2024-03-13T07:15:16.072 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-13T07:15:16.072 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} --verbose --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993 -- /home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993/orig.config.yaml 2024-03-13T07:15:16.279 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T07:15:16.359 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993 branch: wip-yuri8-testing-2024-03-11-1138 description: rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} email: yweinste@redhat.com first_in_suite: false job_id: '7594993' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri8-testing-2024-03-11-1138 ceph: conf: client: debug monc: 20 debug ms: 1 debug objecter: 20 debug rados: 20 global: mon client directed command retry: 5 mon election default strategy: 1 ms inject socket failures: 5000 ms type: async osd heartbeat grace: 80 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osd crush smoke test: false mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 mon warn on pool no app: false paxos service trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 1 debug bluestore: 1 debug monc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd class default list: '*' osd class load list: '*' osd client watch timeout: 120 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd fast shutdown: false osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd memory target: 1610612736 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - reached quota - overall HEALTH_ - \(CACHE_POOL_NO_HIT_SET\) - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(SLOW_OPS\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(OBJECT_MISPLACED\) - slow request - overall HEALTH_ - \(PG_ - \(MON_DOWN\) - \(SLOW_OPS\) - slow request sha1: b7c579c9c3ed531070a6aa96f74f80b53f80de32 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: debuginfo: true flavor: default sha1: b7c579c9c3ed531070a6aa96f74f80b53f80de32 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri8-testing-2024-03-11-1138 sha1: b7c579c9c3ed531070a6aa96f74f80b53f80de32 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 1266 sha1: b7c579c9c3ed531070a6aa96f74f80b53f80de32 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri8-testing-2024-03-11-1138 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: b7c579c9c3ed531070a6aa96f74f80b53f80de32 targets: smithi132.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEFZB1Zl15oaEhsQoQkm4aIPLf9pprFtx9T0xmH3GkM7BJHh9G1TANIlJ/xMCqXsZ7bEm0hx6mFdZpYK3/Lq3jE= smithi195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3eaO3daDzXfRTyTOLae0YzBR4WELsoKJuVa+GGGYjFqxjdnHE6e5I67oohVXUtMNTEGwJME1e7kHpeYpVSJXs= tasks: - install: null - ceph: null - workunit: clients: client.0: - rados/test.sh env: ALLOW_TIMEOUTS: '1' timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-12_18:29:22 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-13T07:15:16.359 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa; will attempt to use it 2024-03-13T07:15:16.360 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks 2024-03-13T07:15:16.360 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-13T07:15:16.363 INFO:teuthology.task.internal:Checking packages... 2024-03-13T07:15:16.393 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b7c579c9c3ed531070a6aa96f74f80b53f80de32' 2024-03-13T07:15:16.393 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-13T07:15:16.394 INFO:teuthology.packaging:ref: None 2024-03-13T07:15:16.394 INFO:teuthology.packaging:tag: None 2024-03-13T07:15:16.394 INFO:teuthology.packaging:branch: wip-yuri8-testing-2024-03-11-1138 2024-03-13T07:15:16.394 INFO:teuthology.packaging:sha1: b7c579c9c3ed531070a6aa96f74f80b53f80de32 2024-03-13T07:15:16.394 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri8-testing-2024-03-11-1138 2024-03-13T07:15:16.611 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2007.gb7c579c9 2024-03-13T07:15:16.613 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-13T07:15:16.632 INFO:teuthology.task.internal:no buildpackages task found 2024-03-13T07:15:16.632 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-13T07:15:16.647 INFO:teuthology.task.internal:Saving configuration 2024-03-13T07:15:16.662 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-13T07:15:16.671 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-13T07:15:16.699 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-13 07:10:22.440330', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEFZB1Zl15oaEhsQoQkm4aIPLf9pprFtx9T0xmH3GkM7BJHh9G1TANIlJ/xMCqXsZ7bEm0hx6mFdZpYK3/Lq3jE='} 2024-03-13T07:15:16.722 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-13 07:10:22.441691', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3eaO3daDzXfRTyTOLae0YzBR4WELsoKJuVa+GGGYjFqxjdnHE6e5I67oohVXUtMNTEGwJME1e7kHpeYpVSJXs='} 2024-03-13T07:15:16.722 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-13T07:15:16.727 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-03-13T07:15:16.728 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-03-13T07:15:16.728 INFO:teuthology.run_tasks:Running task console_log... 2024-03-13T07:15:16.838 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbb5a8dff70>, signals=[15]) 2024-03-13T07:15:16.838 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-13T07:15:16.844 INFO:teuthology.task.internal:Opening connections... 2024-03-13T07:15:16.845 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi132.front.sepia.ceph.com 2024-03-13T07:15:16.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T07:15:16.939 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi195.front.sepia.ceph.com 2024-03-13T07:15:16.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T07:15:17.033 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-13T07:15:17.040 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2024-03-13T07:15:17.057 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2024-03-13T07:15:17.058 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2024-03-13T07:15:17.114 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="CentOS Stream" 2024-03-13T07:15:17.114 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="9" 2024-03-13T07:15:17.115 INFO:teuthology.orchestra.run.smithi132.stdout:ID="centos" 2024-03-13T07:15:17.115 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="rhel fedora" 2024-03-13T07:15:17.115 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="9" 2024-03-13T07:15:17.115 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el9" 2024-03-13T07:15:17.115 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-13T07:15:17.115 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2024-03-13T07:15:17.115 INFO:teuthology.orchestra.run.smithi132.stdout:LOGO="fedora-logo-icon" 2024-03-13T07:15:17.115 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-13T07:15:17.115 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://centos.org/" 2024-03-13T07:15:17.115 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T07:15:17.115 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-13T07:15:17.115 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-13T07:15:17.116 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2024-03-13T07:15:17.141 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2024-03-13T07:15:17.159 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2024-03-13T07:15:17.159 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2024-03-13T07:15:17.220 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream" 2024-03-13T07:15:17.220 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="9" 2024-03-13T07:15:17.220 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos" 2024-03-13T07:15:17.220 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora" 2024-03-13T07:15:17.221 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="9" 2024-03-13T07:15:17.221 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el9" 2024-03-13T07:15:17.221 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-13T07:15:17.221 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31" 2024-03-13T07:15:17.221 INFO:teuthology.orchestra.run.smithi195.stdout:LOGO="fedora-logo-icon" 2024-03-13T07:15:17.221 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-13T07:15:17.221 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/" 2024-03-13T07:15:17.221 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T07:15:17.221 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-13T07:15:17.221 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-13T07:15:17.221 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2024-03-13T07:15:17.241 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-13T07:15:17.250 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-13T07:15:17.255 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-13T07:15:17.255 DEBUG:teuthology.orchestra.run.smithi132:> test '!' -e /home/ubuntu/cephtest 2024-03-13T07:15:17.258 DEBUG:teuthology.orchestra.run.smithi195:> test '!' -e /home/ubuntu/cephtest 2024-03-13T07:15:17.280 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-13T07:15:17.287 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-13T07:15:17.287 DEBUG:teuthology.orchestra.run.smithi132:> test -z $(ls -A /var/lib/ceph) 2024-03-13T07:15:17.314 DEBUG:teuthology.orchestra.run.smithi195:> test -z $(ls -A /var/lib/ceph) 2024-03-13T07:15:17.424 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-13T07:15:17.478 INFO:teuthology.run_tasks:Running task kernel... 2024-03-13T07:15:17.492 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-13T07:15:17.492 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-13T07:15:17.493 DEBUG:teuthology.orchestra.run.smithi132:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-13T07:15:17.493 DEBUG:teuthology.orchestra.run.smithi195:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-13T07:15:17.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T07:15:17.509 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2024-03-13T07:15:17.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T07:15:17.511 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-03-13T07:15:17.567 INFO:teuthology.orchestra.run.smithi132.stdout:5.14.0-361.el9.x86_64 2024-03-13T07:15:17.567 INFO:teuthology.task.kernel:Running kernel on smithi132: 5.14.0-361.el9.x86_64 2024-03-13T07:15:17.567 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum install -y kernel 2024-03-13T07:15:17.570 INFO:teuthology.orchestra.run.smithi195.stdout:5.14.0-361.el9.x86_64 2024-03-13T07:15:17.571 INFO:teuthology.task.kernel:Running kernel on smithi195: 5.14.0-361.el9.x86_64 2024-03-13T07:15:17.571 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum install -y kernel 2024-03-13T07:15:19.760 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-13T07:15:19.970 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS-9 - AppStream 9.3 MB/s | 6.7 MB 00:00 2024-03-13T07:15:21.427 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2024-03-13T07:15:21.646 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2024-03-13T07:15:22.869 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - BaseOS 8.9 MB/s | 8.0 MB 00:00 2024-03-13T07:15:26.734 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - BaseOS 1.6 MB/s | 8.0 MB 00:04 2024-03-13T07:15:28.685 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-03-13T07:15:34.077 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.1 MB 00:01 2024-03-13T07:15:35.536 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 15 kB 00:00 2024-03-13T07:15:37.065 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-13T07:15:39.485 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-03-13T07:15:42.030 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-13T07:15:44.211 INFO:teuthology.orchestra.run.smithi132.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-13T07:15:44.211 INFO:teuthology.orchestra.run.smithi132.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-13T07:15:44.350 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:15:44.356 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:15:44.356 INFO:teuthology.orchestra.run.smithi132.stdout: Package Architecture Version Repository Size 2024-03-13T07:15:44.356 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:15:44.356 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:15:44.357 INFO:teuthology.orchestra.run.smithi132.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-13T07:15:44.357 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-03-13T07:15:44.357 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-13T07:15:44.357 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-13T07:15:44.357 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-13T07:15:44.357 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:15:44.357 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:15:44.357 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:15:44.357 INFO:teuthology.orchestra.run.smithi132.stdout:Install 4 Packages 2024-03-13T07:15:44.357 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:15:44.358 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 100 M 2024-03-13T07:15:44.358 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 123 M 2024-03-13T07:15:44.358 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:15:44.689 INFO:teuthology.orchestra.run.smithi132.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 21 MB/s | 6.0 MB 00:00 2024-03-13T07:15:44.719 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.1 MB 00:01 2024-03-13T07:15:45.022 INFO:teuthology.orchestra.run.smithi132.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 34 MB/s | 21 MB 00:00 2024-03-13T07:15:45.648 INFO:teuthology.orchestra.run.smithi132.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 32 MB/s | 39 MB 00:01 2024-03-13T07:15:45.973 INFO:teuthology.orchestra.run.smithi132.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 26 MB/s | 33 MB 00:01 2024-03-13T07:15:45.974 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:15:45.974 INFO:teuthology.orchestra.run.smithi132.stdout:Total 62 MB/s | 100 MB 00:01 2024-03-13T07:15:46.269 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 15 kB 00:00 2024-03-13T07:15:46.866 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:15:47.234 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:15:47.234 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:15:47.746 INFO:teuthology.orchestra.run.smithi195.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-13T07:15:48.577 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:15:48.578 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:15:51.268 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:15:51.911 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-13T07:15:52.042 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-13T07:15:52.813 INFO:teuthology.orchestra.run.smithi195.stdout:lab-extras 8.3 kB/s | 1.7 kB 00:00 2024-03-13T07:15:54.790 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-13T07:15:54.953 INFO:teuthology.orchestra.run.smithi195.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-13T07:15:54.953 INFO:teuthology.orchestra.run.smithi195.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-13T07:15:54.971 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-13T07:15:55.060 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:15:55.076 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:15:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: Package Architecture Version Repository Size 2024-03-13T07:15:55.076 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:15:55.076 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:15:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-13T07:15:55.076 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-03-13T07:15:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-13T07:15:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-13T07:15:55.077 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-13T07:15:55.077 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:15:55.077 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:15:55.077 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:15:55.077 INFO:teuthology.orchestra.run.smithi195.stdout:Install 4 Packages 2024-03-13T07:15:55.077 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:15:55.078 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 100 M 2024-03-13T07:15:55.078 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 123 M 2024-03-13T07:15:55.078 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:15:55.422 INFO:teuthology.orchestra.run.smithi195.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 21 MB/s | 6.0 MB 00:00 2024-03-13T07:15:55.764 INFO:teuthology.orchestra.run.smithi195.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 34 MB/s | 21 MB 00:00 2024-03-13T07:15:56.148 INFO:teuthology.orchestra.run.smithi195.stdout:(3/4): kernel-modules-core-5.14.0-427.el9.x86_6 46 MB/s | 33 MB 00:00 2024-03-13T07:15:56.565 INFO:teuthology.orchestra.run.smithi195.stdout:(4/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 28 MB/s | 39 MB 00:01 2024-03-13T07:15:56.565 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:15:56.565 INFO:teuthology.orchestra.run.smithi195.stdout:Total 67 MB/s | 100 MB 00:01 2024-03-13T07:15:57.447 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:15:57.821 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:15:57.821 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:15:59.162 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:15:59.162 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:15:59.724 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-13T07:15:59.848 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-13T07:16:01.870 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:16:02.530 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-13T07:16:02.665 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-13T07:16:04.618 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-13T07:16:05.425 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-13T07:16:05.600 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-13T07:16:10.338 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-13T07:16:10.470 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-13T07:16:15.263 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-13T07:16:34.090 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-13T07:16:34.090 INFO:teuthology.orchestra.run.smithi132.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-13T07:16:34.090 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:16:34.176 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-13T07:16:34.632 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-13T07:16:34.632 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-13T07:16:34.633 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-13T07:16:34.633 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-13T07:16:35.040 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-13T07:16:35.040 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:16:35.040 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:16:35.040 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-13T07:16:35.040 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-13T07:16:35.040 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-13T07:16:35.040 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-13T07:16:35.040 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:16:35.040 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:16:35.580 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q kernel | sort -rV | head -n 1 2024-03-13T07:16:35.732 INFO:teuthology.orchestra.run.smithi132.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-13T07:16:35.733 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-13T07:16:35.733 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-13T07:16:35.733 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-13T07:16:35.733 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-13T07:16:35.733 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-13T07:16:35.733 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi132.front.sepia.ceph.com, path=None, version=distro) 2024-03-13T07:16:35.733 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q kernel | sort -rV | head -n 1 2024-03-13T07:16:35.856 INFO:teuthology.orchestra.run.smithi132.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-13T07:16:35.856 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-13T07:16:35.856 DEBUG:teuthology.orchestra.run.smithi132:> sudo rpm -qi grub2-tools 2024-03-13T07:16:35.900 INFO:teuthology.orchestra.run.smithi132.stdout:Name : grub2-tools 2024-03-13T07:16:35.900 INFO:teuthology.orchestra.run.smithi132.stdout:Epoch : 1 2024-03-13T07:16:35.900 INFO:teuthology.orchestra.run.smithi132.stdout:Version : 2.06 2024-03-13T07:16:35.900 INFO:teuthology.orchestra.run.smithi132.stdout:Release : 68.el9 2024-03-13T07:16:35.901 INFO:teuthology.orchestra.run.smithi132.stdout:Architecture: x86_64 2024-03-13T07:16:35.901 INFO:teuthology.orchestra.run.smithi132.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-13T07:16:35.901 INFO:teuthology.orchestra.run.smithi132.stdout:Group : Unspecified 2024-03-13T07:16:35.901 INFO:teuthology.orchestra.run.smithi132.stdout:Size : 8264254 2024-03-13T07:16:35.901 INFO:teuthology.orchestra.run.smithi132.stdout:License : GPLv3+ 2024-03-13T07:16:35.901 INFO:teuthology.orchestra.run.smithi132.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-13T07:16:35.902 INFO:teuthology.orchestra.run.smithi132.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-13T07:16:35.902 INFO:teuthology.orchestra.run.smithi132.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-13T07:16:35.902 INFO:teuthology.orchestra.run.smithi132.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-13T07:16:35.902 INFO:teuthology.orchestra.run.smithi132.stdout:Packager : builder@centos.org 2024-03-13T07:16:35.902 INFO:teuthology.orchestra.run.smithi132.stdout:Vendor : CentOS 2024-03-13T07:16:35.902 INFO:teuthology.orchestra.run.smithi132.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-13T07:16:35.902 INFO:teuthology.orchestra.run.smithi132.stdout:Summary : Support tools for GRUB. 2024-03-13T07:16:35.903 INFO:teuthology.orchestra.run.smithi132.stdout:Description : 2024-03-13T07:16:35.903 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:16:35.903 INFO:teuthology.orchestra.run.smithi132.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-13T07:16:35.903 INFO:teuthology.orchestra.run.smithi132.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-13T07:16:35.903 INFO:teuthology.orchestra.run.smithi132.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-13T07:16:35.903 INFO:teuthology.orchestra.run.smithi132.stdout:hardware devices. 2024-03-13T07:16:35.903 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:16:35.904 INFO:teuthology.orchestra.run.smithi132.stdout:This subpackage provides tools for support of all platforms. 2024-03-13T07:16:35.905 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-13T07:16:35.905 INFO:teuthology.task.kernel:Updating grub on smithi132 to boot 5.14.0-427.el9.x86_64 2024-03-13T07:16:35.905 DEBUG:teuthology.orchestra.run.smithi132:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-13T07:16:36.477 INFO:teuthology.orchestra.run.smithi132.stderr:Generating grub configuration file ... 2024-03-13T07:16:37.951 INFO:teuthology.orchestra.run.smithi132.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-13T07:16:37.988 INFO:teuthology.orchestra.run.smithi132.stderr:done 2024-03-13T07:16:37.991 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-03-13T07:16:38.006 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.xjcRURV5cN 2024-03-13T07:16:38.007 DEBUG:teuthology.orchestra.run.smithi132:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.xjcRURV5cN 2024-03-13T07:16:38.074 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.xjcRURV5cN 2024-03-13T07:16:38.210 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.xjcRURV5cN is 6KB 2024-03-13T07:16:38.259 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.xjcRURV5cN 2024-03-13T07:16:38.274 DEBUG:teuthology.orchestra.run.smithi132:> sudo /bin/ls /boot/loader/entries || true 2024-03-13T07:16:38.338 INFO:teuthology.orchestra.run.smithi132.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-13T07:16:38.339 INFO:teuthology.orchestra.run.smithi132.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-13T07:16:38.339 INFO:teuthology.orchestra.run.smithi132.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-13T07:16:38.339 INFO:teuthology.orchestra.run.smithi132.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-13T07:16:38.340 DEBUG:teuthology.orchestra.run.smithi132:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-13T07:16:38.507 DEBUG:teuthology.orchestra.run.smithi132:> sudo shutdown -r now 2024-03-13T07:16:44.826 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-13T07:16:44.826 INFO:teuthology.orchestra.run.smithi195.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-13T07:16:44.826 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:16:44.912 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-13T07:16:45.353 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-13T07:16:45.353 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-13T07:16:45.353 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-13T07:16:45.354 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-13T07:16:45.820 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-13T07:16:45.820 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:16:45.820 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:16:45.820 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-13T07:16:45.821 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-13T07:16:45.821 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-13T07:16:45.821 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-13T07:16:45.821 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:16:45.821 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:16:46.363 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q kernel | sort -rV | head -n 1 2024-03-13T07:16:46.508 INFO:teuthology.orchestra.run.smithi195.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-13T07:16:46.508 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-13T07:16:46.508 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-13T07:16:46.508 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-13T07:16:46.508 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-13T07:16:46.509 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-13T07:16:46.509 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi195.front.sepia.ceph.com, path=None, version=distro) 2024-03-13T07:16:46.509 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q kernel | sort -rV | head -n 1 2024-03-13T07:16:46.629 INFO:teuthology.orchestra.run.smithi195.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-13T07:16:46.629 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-13T07:16:46.629 DEBUG:teuthology.orchestra.run.smithi195:> sudo rpm -qi grub2-tools 2024-03-13T07:16:46.663 INFO:teuthology.orchestra.run.smithi195.stdout:Name : grub2-tools 2024-03-13T07:16:46.663 INFO:teuthology.orchestra.run.smithi195.stdout:Epoch : 1 2024-03-13T07:16:46.663 INFO:teuthology.orchestra.run.smithi195.stdout:Version : 2.06 2024-03-13T07:16:46.663 INFO:teuthology.orchestra.run.smithi195.stdout:Release : 68.el9 2024-03-13T07:16:46.663 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: x86_64 2024-03-13T07:16:46.663 INFO:teuthology.orchestra.run.smithi195.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-13T07:16:46.663 INFO:teuthology.orchestra.run.smithi195.stdout:Group : Unspecified 2024-03-13T07:16:46.663 INFO:teuthology.orchestra.run.smithi195.stdout:Size : 8264254 2024-03-13T07:16:46.664 INFO:teuthology.orchestra.run.smithi195.stdout:License : GPLv3+ 2024-03-13T07:16:46.664 INFO:teuthology.orchestra.run.smithi195.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-13T07:16:46.664 INFO:teuthology.orchestra.run.smithi195.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-13T07:16:46.664 INFO:teuthology.orchestra.run.smithi195.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-13T07:16:46.664 INFO:teuthology.orchestra.run.smithi195.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-13T07:16:46.664 INFO:teuthology.orchestra.run.smithi195.stdout:Packager : builder@centos.org 2024-03-13T07:16:46.664 INFO:teuthology.orchestra.run.smithi195.stdout:Vendor : CentOS 2024-03-13T07:16:46.664 INFO:teuthology.orchestra.run.smithi195.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-13T07:16:46.664 INFO:teuthology.orchestra.run.smithi195.stdout:Summary : Support tools for GRUB. 2024-03-13T07:16:46.664 INFO:teuthology.orchestra.run.smithi195.stdout:Description : 2024-03-13T07:16:46.664 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:16:46.664 INFO:teuthology.orchestra.run.smithi195.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-13T07:16:46.664 INFO:teuthology.orchestra.run.smithi195.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-13T07:16:46.664 INFO:teuthology.orchestra.run.smithi195.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-13T07:16:46.664 INFO:teuthology.orchestra.run.smithi195.stdout:hardware devices. 2024-03-13T07:16:46.665 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:16:46.665 INFO:teuthology.orchestra.run.smithi195.stdout:This subpackage provides tools for support of all platforms. 2024-03-13T07:16:46.665 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-13T07:16:46.666 INFO:teuthology.task.kernel:Updating grub on smithi195 to boot 5.14.0-427.el9.x86_64 2024-03-13T07:16:46.666 DEBUG:teuthology.orchestra.run.smithi195:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-13T07:16:47.277 INFO:teuthology.orchestra.run.smithi195.stderr:Generating grub configuration file ... 2024-03-13T07:16:48.596 INFO:teuthology.orchestra.run.smithi195.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-13T07:16:48.633 INFO:teuthology.orchestra.run.smithi195.stderr:done 2024-03-13T07:16:48.726 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-03-13T07:16:48.754 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.REalD4TyWe 2024-03-13T07:16:48.755 DEBUG:teuthology.orchestra.run.smithi195:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.REalD4TyWe 2024-03-13T07:16:48.826 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.REalD4TyWe 2024-03-13T07:16:48.974 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.REalD4TyWe is 6KB 2024-03-13T07:16:49.031 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.REalD4TyWe 2024-03-13T07:16:49.049 DEBUG:teuthology.orchestra.run.smithi195:> sudo /bin/ls /boot/loader/entries || true 2024-03-13T07:16:49.124 INFO:teuthology.orchestra.run.smithi195.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-13T07:16:49.124 INFO:teuthology.orchestra.run.smithi195.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-13T07:16:49.125 INFO:teuthology.orchestra.run.smithi195.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-13T07:16:49.125 INFO:teuthology.orchestra.run.smithi195.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-13T07:16:49.126 DEBUG:teuthology.orchestra.run.smithi195:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-13T07:16:49.267 DEBUG:teuthology.orchestra.run.smithi195:> sudo shutdown -r now 2024-03-13T07:17:08.526 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-13T07:17:08.527 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-13T07:17:08.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T07:17:19.278 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-13T07:17:19.279 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-03-13T07:17:19.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T07:17:26.943 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.132 2024-03-13T07:17:35.950 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-13T07:17:35.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T07:17:37.698 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.195 2024-03-13T07:17:46.708 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-03-13T07:17:46.708 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T07:18:35.956 DEBUG:teuthology.orchestra.remote:timed out 2024-03-13T07:18:46.718 DEBUG:teuthology.orchestra.remote:timed out 2024-03-13T07:18:47.959 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-13T07:18:47.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T07:18:48.603 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-03-13T07:18:48.980 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-13T07:18:48.980 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-427.el9.x86_64"... 2024-03-13T07:18:48.981 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2024-03-13T07:18:48.996 INFO:teuthology.orchestra.run.smithi132.stdout:5.14.0-427.el9.x86_64 2024-03-13T07:18:48.996 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-13T07:18:48.996 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-13T07:18:48.996 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-13T07:18:49.997 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-13T07:18:49.997 DEBUG:teuthology.orchestra.run.smithi132:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-13T07:18:50.056 INFO:teuthology.orchestra.run.smithi132.stdout:ttyS1 2024-03-13T07:18:50.098 DEBUG:teuthology.parallel:result is None 2024-03-13T07:18:58.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-03-13T07:18:58.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T07:18:59.190 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-03-13T07:18:59.590 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-03-13T07:18:59.590 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-427.el9.x86_64"... 2024-03-13T07:18:59.590 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-03-13T07:18:59.612 INFO:teuthology.orchestra.run.smithi195.stdout:5.14.0-427.el9.x86_64 2024-03-13T07:18:59.612 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-13T07:18:59.612 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-13T07:18:59.612 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-13T07:19:00.613 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-13T07:19:00.614 DEBUG:teuthology.orchestra.run.smithi195:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-13T07:19:00.690 INFO:teuthology.orchestra.run.smithi195.stdout:ttyS1 2024-03-13T07:19:00.724 DEBUG:teuthology.parallel:result is None 2024-03-13T07:19:00.724 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-13T07:19:00.740 INFO:teuthology.task.internal:Creating test directory... 2024-03-13T07:19:00.740 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-13T07:19:00.744 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-13T07:19:00.763 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-13T07:19:00.883 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-13T07:19:00.903 INFO:teuthology.task.internal:Creating archive directory... 2024-03-13T07:19:00.903 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-13T07:19:00.911 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-13T07:19:00.948 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-13T07:19:00.969 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-13T07:19:00.969 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-13T07:19:00.980 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-13T07:19:01.017 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-13T07:19:01.022 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-13T07:19:01.031 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-13T07:19:01.036 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-13T07:19:01.039 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-13T07:19:01.054 INFO:teuthology.task.internal:Configuring sudo... 2024-03-13T07:19:01.054 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-13T07:19:01.058 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-13T07:19:01.106 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-13T07:19:01.121 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-13T07:19:01.121 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-13T07:19:01.129 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-13T07:19:01.164 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-13T07:19:01.232 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-13T07:19:01.277 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-13T07:19:01.361 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-13T07:19:01.404 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:19:01.404 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-13T07:19:01.471 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-13T07:19:01.516 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-13T07:19:01.571 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-13T07:19:01.655 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-13T07:19:01.700 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:19:01.700 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-13T07:19:01.769 DEBUG:teuthology.orchestra.run.smithi132:> sudo service rsyslog restart 2024-03-13T07:19:01.772 DEBUG:teuthology.orchestra.run.smithi195:> sudo service rsyslog restart 2024-03-13T07:19:01.818 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-13T07:19:01.856 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-13T07:19:02.311 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-13T07:19:02.318 INFO:teuthology.task.internal:Starting timer... 2024-03-13T07:19:02.319 INFO:teuthology.run_tasks:Running task pcp... 2024-03-13T07:19:02.329 INFO:teuthology.run_tasks:Running task selinux... 2024-03-13T07:19:02.340 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-03-13T07:19:02.340 DEBUG:teuthology.orchestra.run.smithi132:> sudo service auditd rotate 2024-03-13T07:19:02.396 INFO:teuthology.orchestra.run.smithi132.stdout:Rotating logs: 2024-03-13T07:19:02.398 DEBUG:teuthology.orchestra.run.smithi195:> sudo service auditd rotate 2024-03-13T07:19:02.448 INFO:teuthology.orchestra.run.smithi195.stdout:Rotating logs: 2024-03-13T07:19:02.451 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-13T07:19:02.451 DEBUG:teuthology.orchestra.run.smithi132:> /usr/sbin/getenforce 2024-03-13T07:19:02.483 INFO:teuthology.orchestra.run.smithi132.stdout:Permissive 2024-03-13T07:19:02.484 DEBUG:teuthology.orchestra.run.smithi195:> /usr/sbin/getenforce 2024-03-13T07:19:02.510 INFO:teuthology.orchestra.run.smithi195.stdout:Permissive 2024-03-13T07:19:02.510 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi132.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi195.front.sepia.ceph.com': 'permissive'} 2024-03-13T07:19:02.510 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-13T07:19:02.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T07:19:02.553 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-13T07:19:02.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T07:19:02.583 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-13T07:19:02.583 DEBUG:teuthology.orchestra.run.smithi132:> sudo /usr/sbin/setenforce permissive 2024-03-13T07:19:02.636 DEBUG:teuthology.orchestra.run.smithi195:> sudo /usr/sbin/setenforce permissive 2024-03-13T07:19:02.670 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-13T07:19:02.680 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-13T07:19:02.701 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-13T07:19:02.702 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi132.front.sepia.ceph.com,smithi195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-13T07:23:17.737 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi132.front.sepia.ceph.com'), Remote(name='ubuntu@smithi195.front.sepia.ceph.com')] 2024-03-13T07:23:17.738 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-13T07:23:17.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T07:23:17.834 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-03-13T07:23:17.913 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-13T07:23:17.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-03-13T07:23:17.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T07:23:18.007 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-03-13T07:23:18.086 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-03-13T07:23:18.087 INFO:teuthology.run_tasks:Running task clock... 2024-03-13T07:23:18.098 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-13T07:23:18.098 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-13T07:23:18.098 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-13T07:23:18.102 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-13T07:23:18.102 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-13T07:23:18.131 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-13T07:23:18.147 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-13T07:23:18.159 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-13T07:23:18.176 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: ntpd: command not found 2024-03-13T07:23:18.176 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-13T07:23:18.189 INFO:teuthology.orchestra.run.smithi132.stdout:506 Cannot talk to daemon 2024-03-13T07:23:18.206 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-13T07:23:18.206 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: ntpd: command not found 2024-03-13T07:23:18.220 INFO:teuthology.orchestra.run.smithi195.stdout:506 Cannot talk to daemon 2024-03-13T07:23:18.222 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-13T07:23:18.236 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-13T07:23:18.253 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-13T07:23:18.276 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 1: ntpq: command not found 2024-03-13T07:23:18.280 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-13T07:23:18.280 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-03-13T07:23:18.280 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T07:23:18.280 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T07:23:18.280 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T07:23:18.280 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T07:23:18.311 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2024-03-13T07:23:18.315 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-13T07:23:18.315 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2024-03-13T07:23:18.315 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T07:23:18.315 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T07:23:18.315 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T07:23:18.315 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T07:23:18.316 INFO:teuthology.run_tasks:Running task install... 2024-03-13T07:23:18.324 DEBUG:teuthology.task.install:project ceph 2024-03-13T07:23:18.325 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'b7c579c9c3ed531070a6aa96f74f80b53f80de32'}} 2024-03-13T07:23:18.325 DEBUG:teuthology.task.install:config {'debuginfo': True, 'flavor': 'default', 'sha1': 'b7c579c9c3ed531070a6aa96f74f80b53f80de32'} 2024-03-13T07:23:18.325 INFO:teuthology.task.install:Using flavor: default 2024-03-13T07:23:18.330 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2024-03-13T07:23:18.331 INFO:teuthology.task.install:extra packages: [] 2024-03-13T07:23:18.331 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b7c579c9c3ed531070a6aa96f74f80b53f80de32', 'tag': None, 'wait_for_package': False} 2024-03-13T07:23:18.332 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b7c579c9c3ed531070a6aa96f74f80b53f80de32 2024-03-13T07:23:18.334 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b7c579c9c3ed531070a6aa96f74f80b53f80de32', 'tag': None, 'wait_for_package': False} 2024-03-13T07:23:18.334 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b7c579c9c3ed531070a6aa96f74f80b53f80de32 2024-03-13T07:23:18.480 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2024-03-11-1138/b7c579c9c3ed531070a6aa96f74f80b53f80de32/centos/9/flavors/default/ 2024-03-13T07:23:18.480 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2007.gb7c579c9 2024-03-13T07:23:18.554 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2024-03-11-1138/b7c579c9c3ed531070a6aa96f74f80b53f80de32/centos/9/flavors/default/ 2024-03-13T07:23:18.554 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2007.gb7c579c9 2024-03-13T07:23:18.591 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2024-03-11-1138/b7c579c9c3ed531070a6aa96f74f80b53f80de32/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2024-03-11-1138/b7c579c9c3ed531070a6aa96f74f80b53f80de32/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2024-03-11-1138/b7c579c9c3ed531070a6aa96f74f80b53f80de32/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-13T07:23:18.592 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:23:18.592 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-13T07:23:18.619 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2024-03-13T07:23:18.620 DEBUG:teuthology.orchestra.run.smithi132:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b7c579c9c3ed531070a6aa96f74f80b53f80de32/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-13T07:23:18.686 DEBUG:teuthology.orchestra.run.smithi132:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-13T07:23:18.724 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2024-03-11-1138/b7c579c9c3ed531070a6aa96f74f80b53f80de32/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2024-03-11-1138/b7c579c9c3ed531070a6aa96f74f80b53f80de32/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2024-03-11-1138/b7c579c9c3ed531070a6aa96f74f80b53f80de32/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-13T07:23:18.724 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:23:18.724 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-13T07:23:18.753 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2024-03-13T07:23:18.753 DEBUG:teuthology.orchestra.run.smithi195:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b7c579c9c3ed531070a6aa96f74f80b53f80de32/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-13T07:23:18.766 DEBUG:teuthology.orchestra.run.smithi132:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-13T07:23:18.821 DEBUG:teuthology.orchestra.run.smithi195:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-13T07:23:18.832 INFO:teuthology.orchestra.run.smithi132.stdout:check_obsoletes = 1 2024-03-13T07:23:18.834 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum clean all 2024-03-13T07:23:18.901 DEBUG:teuthology.orchestra.run.smithi195:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-13T07:23:18.967 INFO:teuthology.orchestra.run.smithi195.stdout:check_obsoletes = 1 2024-03-13T07:23:18.968 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum clean all 2024-03-13T07:23:19.078 INFO:teuthology.orchestra.run.smithi132.stdout:74 files removed 2024-03-13T07:23:19.104 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-radosgw 2024-03-13T07:23:19.209 INFO:teuthology.orchestra.run.smithi195.stdout:74 files removed 2024-03-13T07:23:19.236 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-radosgw 2024-03-13T07:23:19.836 INFO:teuthology.orchestra.run.smithi132.stdout:ceph packages for x86_64 198 kB/s | 82 kB 00:00 2024-03-13T07:23:19.962 INFO:teuthology.orchestra.run.smithi195.stdout:ceph packages for x86_64 202 kB/s | 82 kB 00:00 2024-03-13T07:23:20.204 INFO:teuthology.orchestra.run.smithi132.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-03-13T07:23:20.345 INFO:teuthology.orchestra.run.smithi195.stdout:ceph noarch packages 30 kB/s | 10 kB 00:00 2024-03-13T07:23:20.579 INFO:teuthology.orchestra.run.smithi132.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-03-13T07:23:20.696 INFO:teuthology.orchestra.run.smithi195.stdout:ceph source packages 6.1 kB/s | 1.9 kB 00:00 2024-03-13T07:23:20.981 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-03-13T07:23:21.100 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-03-13T07:23:22.465 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - BaseOS 5.5 MB/s | 2.0 MB 00:00 2024-03-13T07:23:22.643 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.0 MB 00:00 2024-03-13T07:23:23.256 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.0 MB 00:00 2024-03-13T07:23:23.517 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-03-13T07:23:25.016 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-03-13T07:23:25.426 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-03-13T07:23:29.400 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - CRB 6.6 MB/s | 6.1 MB 00:00 2024-03-13T07:23:29.952 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.1 MB 00:01 2024-03-13T07:23:31.017 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 15 kB 00:00 2024-03-13T07:23:31.394 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 15 kB 00:00 2024-03-13T07:23:31.844 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 29 MB/s | 21 MB 00:00 2024-03-13T07:23:32.154 INFO:teuthology.orchestra.run.smithi195.stdout:Extra Packages for Enterprise Linux 32 MB/s | 21 MB 00:00 2024-03-13T07:23:36.760 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-03-13T07:23:37.046 INFO:teuthology.orchestra.run.smithi195.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-13T07:23:39.322 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:23:39.333 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:23:39.333 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-03-13T07:23:39.333 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:23:39.333 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:23:39.333 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 9.8 M 2024-03-13T07:23:39.333 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrading: 2024-03-13T07:23:39.333 INFO:teuthology.orchestra.run.smithi132.stdout: librados2 x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 3.4 M 2024-03-13T07:23:39.333 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1 x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 3.1 M 2024-03-13T07:23:39.333 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-03-13T07:23:39.333 INFO:teuthology.orchestra.run.smithi132.stdout: boost-program-options 2024-03-13T07:23:39.333 INFO:teuthology.orchestra.run.smithi132.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-13T07:23:39.334 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 5.4 M 2024-03-13T07:23:39.334 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 20 M 2024-03-13T07:23:39.334 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 25 k 2024-03-13T07:23:39.334 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-13T07:23:39.334 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-13T07:23:39.334 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-13T07:23:39.334 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2 x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 745 k 2024-03-13T07:23:39.334 INFO:teuthology.orchestra.run.smithi132.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-13T07:23:39.334 INFO:teuthology.orchestra.run.smithi132.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-13T07:23:39.334 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-13T07:23:39.334 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1 x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 501 k 2024-03-13T07:23:39.334 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-13T07:23:39.335 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2 x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 5.2 M 2024-03-13T07:23:39.335 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-13T07:23:39.335 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-13T07:23:39.335 INFO:teuthology.orchestra.run.smithi132.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-13T07:23:39.335 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse 2024-03-13T07:23:39.335 INFO:teuthology.orchestra.run.smithi132.stdout: x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 45 k 2024-03-13T07:23:39.335 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 131 k 2024-03-13T07:23:39.335 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 162 k 2024-03-13T07:23:39.335 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-13T07:23:39.335 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 320 k 2024-03-13T07:23:39.335 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 298 k 2024-03-13T07:23:39.335 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 99 k 2024-03-13T07:23:39.335 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-13T07:23:39.335 INFO:teuthology.orchestra.run.smithi132.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-13T07:23:39.336 INFO:teuthology.orchestra.run.smithi132.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-13T07:23:39.336 INFO:teuthology.orchestra.run.smithi132.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-13T07:23:39.336 INFO:teuthology.orchestra.run.smithi132.stdout:Installing weak dependencies: 2024-03-13T07:23:39.336 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-13T07:23:39.336 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:23:39.336 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:23:39.336 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:23:39.336 INFO:teuthology.orchestra.run.smithi132.stdout:Install 30 Packages 2024-03-13T07:23:39.336 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrade 2 Packages 2024-03-13T07:23:39.336 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:23:39.337 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 58 M 2024-03-13T07:23:39.337 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:23:39.574 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:23:39.588 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:23:39.588 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-03-13T07:23:39.588 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:23:39.589 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:23:39.589 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 9.8 M 2024-03-13T07:23:39.589 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrading: 2024-03-13T07:23:39.589 INFO:teuthology.orchestra.run.smithi195.stdout: librados2 x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 3.4 M 2024-03-13T07:23:39.589 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1 x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 3.1 M 2024-03-13T07:23:39.589 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-03-13T07:23:39.589 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options 2024-03-13T07:23:39.589 INFO:teuthology.orchestra.run.smithi195.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-13T07:23:39.589 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 5.4 M 2024-03-13T07:23:39.589 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 20 M 2024-03-13T07:23:39.589 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 25 k 2024-03-13T07:23:39.589 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-13T07:23:39.589 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-13T07:23:39.590 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-13T07:23:39.590 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2 x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 745 k 2024-03-13T07:23:39.590 INFO:teuthology.orchestra.run.smithi195.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-13T07:23:39.590 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-13T07:23:39.590 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-13T07:23:39.590 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1 x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 501 k 2024-03-13T07:23:39.590 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-13T07:23:39.590 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2 x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 5.2 M 2024-03-13T07:23:39.590 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-13T07:23:39.590 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-13T07:23:39.590 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-13T07:23:39.590 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse 2024-03-13T07:23:39.590 INFO:teuthology.orchestra.run.smithi195.stdout: x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 45 k 2024-03-13T07:23:39.591 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 131 k 2024-03-13T07:23:39.591 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 162 k 2024-03-13T07:23:39.591 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-13T07:23:39.591 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 320 k 2024-03-13T07:23:39.591 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 298 k 2024-03-13T07:23:39.591 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 99 k 2024-03-13T07:23:39.591 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-13T07:23:39.591 INFO:teuthology.orchestra.run.smithi195.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-13T07:23:39.591 INFO:teuthology.orchestra.run.smithi195.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-13T07:23:39.591 INFO:teuthology.orchestra.run.smithi195.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-13T07:23:39.591 INFO:teuthology.orchestra.run.smithi195.stdout:Installing weak dependencies: 2024-03-13T07:23:39.591 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-13T07:23:39.591 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:23:39.592 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:23:39.592 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:23:39.592 INFO:teuthology.orchestra.run.smithi195.stdout:Install 30 Packages 2024-03-13T07:23:39.592 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrade 2 Packages 2024-03-13T07:23:39.592 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:23:39.592 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 58 M 2024-03-13T07:23:39.593 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:23:40.261 INFO:teuthology.orchestra.run.smithi132.stdout:(1/32): ceph-base-19.0.0-2007.gb7c579c9.el9.x86 7.1 MB/s | 5.4 MB 00:00 2024-03-13T07:23:40.329 INFO:teuthology.orchestra.run.smithi132.stdout:(2/32): ceph-selinux-19.0.0-2007.gb7c579c9.el9. 377 kB/s | 25 kB 00:00 2024-03-13T07:23:40.479 INFO:teuthology.orchestra.run.smithi132.stdout:(3/32): libcephfs2-19.0.0-2007.gb7c579c9.el9.x8 4.9 MB/s | 745 kB 00:00 2024-03-13T07:23:40.579 INFO:teuthology.orchestra.run.smithi132.stdout:(4/32): libradosstriper1-19.0.0-2007.gb7c579c9. 4.9 MB/s | 501 kB 00:00 2024-03-13T07:23:41.405 INFO:teuthology.orchestra.run.smithi195.stdout:(1/32): ceph-base-19.0.0-2007.gb7c579c9.el9.x86 3.9 MB/s | 5.4 MB 00:01 2024-03-13T07:23:41.429 INFO:teuthology.orchestra.run.smithi132.stdout:(5/32): librgw2-19.0.0-2007.gb7c579c9.el9.x86_6 6.1 MB/s | 5.2 MB 00:00 2024-03-13T07:23:41.447 INFO:teuthology.orchestra.run.smithi195.stdout:(2/32): ceph-selinux-19.0.0-2007.gb7c579c9.el9. 607 kB/s | 25 kB 00:00 2024-03-13T07:23:41.488 INFO:teuthology.orchestra.run.smithi132.stdout:(6/32): python3-ceph-argparse-19.0.0-2007.gb7c5 777 kB/s | 45 kB 00:00 2024-03-13T07:23:41.539 INFO:teuthology.orchestra.run.smithi195.stdout:(3/32): libcephfs2-19.0.0-2007.gb7c579c9.el9.x8 8.0 MB/s | 745 kB 00:00 2024-03-13T07:23:41.614 INFO:teuthology.orchestra.run.smithi195.stdout:(4/32): libradosstriper1-19.0.0-2007.gb7c579c9. 6.5 MB/s | 501 kB 00:00 2024-03-13T07:23:41.688 INFO:teuthology.orchestra.run.smithi132.stdout:(7/32): ceph-common-19.0.0-2007.gb7c579c9.el9.x 9.2 MB/s | 20 MB 00:02 2024-03-13T07:23:41.714 INFO:teuthology.orchestra.run.smithi132.stdout:(8/32): python3-ceph-common-19.0.0-2007.gb7c579 582 kB/s | 131 kB 00:00 2024-03-13T07:23:41.822 INFO:teuthology.orchestra.run.smithi132.stdout:(9/32): ceph-radosgw-19.0.0-2007.gb7c579c9.el9. 4.2 MB/s | 9.8 MB 00:02 2024-03-13T07:23:41.847 INFO:teuthology.orchestra.run.smithi132.stdout:(10/32): python3-cephfs-19.0.0-2007.gb7c579c9.e 1.0 MB/s | 162 kB 00:00 2024-03-13T07:23:41.873 INFO:teuthology.orchestra.run.smithi132.stdout:(11/32): python3-rados-19.0.0-2007.gb7c579c9.el 2.0 MB/s | 320 kB 00:00 2024-03-13T07:23:41.940 INFO:teuthology.orchestra.run.smithi132.stdout:(12/32): python3-rbd-19.0.0-2007.gb7c579c9.el9. 2.5 MB/s | 298 kB 00:00 2024-03-13T07:23:41.965 INFO:teuthology.orchestra.run.smithi132.stdout:(13/32): python3-rgw-19.0.0-2007.gb7c579c9.el9. 847 kB/s | 99 kB 00:00 2024-03-13T07:23:42.015 INFO:teuthology.orchestra.run.smithi195.stdout:(5/32): librgw2-19.0.0-2007.gb7c579c9.el9.x86_6 13 MB/s | 5.2 MB 00:00 2024-03-13T07:23:42.023 INFO:teuthology.orchestra.run.smithi132.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 704 kB/s | 106 kB 00:00 2024-03-13T07:23:42.049 INFO:teuthology.orchestra.run.smithi132.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 159 kB 00:00 2024-03-13T07:23:42.056 INFO:teuthology.orchestra.run.smithi195.stdout:(6/32): python3-ceph-argparse-19.0.0-2007.gb7c5 1.1 MB/s | 45 kB 00:00 2024-03-13T07:23:42.074 INFO:teuthology.orchestra.run.smithi132.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 406 kB/s | 44 kB 00:00 2024-03-13T07:23:42.115 INFO:teuthology.orchestra.run.smithi195.stdout:(7/32): python3-ceph-common-19.0.0-2007.gb7c579 2.2 MB/s | 131 kB 00:00 2024-03-13T07:23:42.124 INFO:teuthology.orchestra.run.smithi132.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 661 kB 00:00 2024-03-13T07:23:42.149 INFO:teuthology.orchestra.run.smithi132.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 554 kB/s | 41 kB 00:00 2024-03-13T07:23:42.174 INFO:teuthology.orchestra.run.smithi195.stdout:(8/32): python3-cephfs-19.0.0-2007.gb7c579c9.el 2.7 MB/s | 162 kB 00:00 2024-03-13T07:23:42.175 INFO:teuthology.orchestra.run.smithi132.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 282 kB 00:00 2024-03-13T07:23:42.241 INFO:teuthology.orchestra.run.smithi195.stdout:(9/32): python3-rados-19.0.0-2007.gb7c579c9.el9 4.7 MB/s | 320 kB 00:00 2024-03-13T07:23:42.300 INFO:teuthology.orchestra.run.smithi195.stdout:(10/32): python3-rbd-19.0.0-2007.gb7c579c9.el9. 5.0 MB/s | 298 kB 00:00 2024-03-13T07:23:42.350 INFO:teuthology.orchestra.run.smithi195.stdout:(11/32): python3-rgw-19.0.0-2007.gb7c579c9.el9. 2.0 MB/s | 99 kB 00:00 2024-03-13T07:23:42.517 INFO:teuthology.orchestra.run.smithi195.stdout:(12/32): ceph-radosgw-19.0.0-2007.gb7c579c9.el9 4.0 MB/s | 9.8 MB 00:02 2024-03-13T07:23:42.550 INFO:teuthology.orchestra.run.smithi195.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 528 kB/s | 106 kB 00:00 2024-03-13T07:23:42.592 INFO:teuthology.orchestra.run.smithi195.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-13T07:23:42.617 INFO:teuthology.orchestra.run.smithi195.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 662 kB/s | 44 kB 00:00 2024-03-13T07:23:42.751 INFO:teuthology.orchestra.run.smithi195.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 661 kB 00:00 2024-03-13T07:23:42.792 INFO:teuthology.orchestra.run.smithi132.stdout:(20/32): qatlib-service-23.11.0-1.el9.x86_64.rp 54 kB/s | 35 kB 00:00 2024-03-13T07:23:42.801 INFO:teuthology.orchestra.run.smithi195.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.5 MB/s | 282 kB 00:00 2024-03-13T07:23:42.827 INFO:teuthology.orchestra.run.smithi195.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 550 kB/s | 41 kB 00:00 2024-03-13T07:23:42.833 INFO:teuthology.orchestra.run.smithi132.stdout:(21/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 68 kB/s | 44 kB 00:00 2024-03-13T07:23:42.977 INFO:teuthology.orchestra.run.smithi195.stdout:(19/32): qatlib-service-23.11.0-1.el9.x86_64.rp 233 kB/s | 35 kB 00:00 2024-03-13T07:23:43.017 INFO:teuthology.orchestra.run.smithi132.stdout:(22/32): qatlib-23.11.0-1.el9.x86_64.rpm 243 kB/s | 216 kB 00:00 2024-03-13T07:23:43.069 INFO:teuthology.orchestra.run.smithi195.stdout:(20/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 486 kB/s | 44 kB 00:00 2024-03-13T07:23:43.093 INFO:teuthology.orchestra.run.smithi132.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.0 MB/s | 309 kB 00:00 2024-03-13T07:23:43.094 INFO:teuthology.orchestra.run.smithi195.stdout:(21/32): qatlib-23.11.0-1.el9.x86_64.rpm 741 kB/s | 216 kB 00:00 2024-03-13T07:23:43.134 INFO:teuthology.orchestra.run.smithi132.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 1.2 MB/s | 49 kB 00:00 2024-03-13T07:23:43.159 INFO:teuthology.orchestra.run.smithi132.stdout:(25/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 183 kB/s | 26 kB 00:00 2024-03-13T07:23:43.193 INFO:teuthology.orchestra.run.smithi132.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-13T07:23:43.260 INFO:teuthology.orchestra.run.smithi132.stdout:(27/32): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-03-13T07:23:43.285 INFO:teuthology.orchestra.run.smithi132.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-03-13T07:23:43.335 INFO:teuthology.orchestra.run.smithi132.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-03-13T07:23:43.386 INFO:teuthology.orchestra.run.smithi132.stdout:(30/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 838 kB 00:00 2024-03-13T07:23:43.453 INFO:teuthology.orchestra.run.smithi195.stdout:(22/32): ceph-common-19.0.0-2007.gb7c579c9.el9. 5.8 MB/s | 20 MB 00:03 2024-03-13T07:23:43.611 INFO:teuthology.orchestra.run.smithi195.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 569 kB/s | 309 kB 00:00 2024-03-13T07:23:43.645 INFO:teuthology.orchestra.run.smithi195.stdout:(24/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 136 kB/s | 26 kB 00:00 2024-03-13T07:23:43.661 INFO:teuthology.orchestra.run.smithi132.stdout:(31/32): librbd1-19.0.0-2007.gb7c579c9.el9.x86_ 9.7 MB/s | 3.1 MB 00:00 2024-03-13T07:23:43.670 INFO:teuthology.orchestra.run.smithi195.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 827 kB/s | 49 kB 00:00 2024-03-13T07:23:43.711 INFO:teuthology.orchestra.run.smithi132.stdout:(32/32): librados2-19.0.0-2007.gb7c579c9.el9.x8 7.9 MB/s | 3.4 MB 00:00 2024-03-13T07:23:43.718 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:23:43.718 INFO:teuthology.orchestra.run.smithi132.stdout:Total 13 MB/s | 58 MB 00:04 2024-03-13T07:23:43.737 INFO:teuthology.orchestra.run.smithi195.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 731 kB/s | 67 kB 00:00 2024-03-13T07:23:43.751 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:23:43.787 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:23:43.787 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:23:43.871 INFO:teuthology.orchestra.run.smithi195.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-03-13T07:23:44.029 INFO:teuthology.orchestra.run.smithi195.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.3 MB/s | 838 kB 00:00 2024-03-13T07:23:44.277 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:23:44.277 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:23:44.313 INFO:teuthology.orchestra.run.smithi195.stdout:(29/32): librados2-19.0.0-2007.gb7c579c9.el9.x8 12 MB/s | 3.4 MB 00:00 2024-03-13T07:23:44.480 INFO:teuthology.orchestra.run.smithi195.stdout:(30/32): librbd1-19.0.0-2007.gb7c579c9.el9.x86_ 19 MB/s | 3.1 MB 00:00 2024-03-13T07:23:44.513 INFO:teuthology.orchestra.run.smithi195.stdout:(31/32): thrift-0.15.0-2.el9.x86_64.rpm 2.5 MB/s | 1.6 MB 00:00 2024-03-13T07:23:44.638 INFO:teuthology.orchestra.run.smithi195.stdout:(32/32): libarrow-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 4.4 MB 00:01 2024-03-13T07:23:44.644 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:23:44.644 INFO:teuthology.orchestra.run.smithi195.stdout:Total 12 MB/s | 58 MB 00:05 2024-03-13T07:23:44.679 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:23:44.717 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:23:44.717 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:23:45.198 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:23:45.198 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:23:45.764 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:23:45.848 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-03-13T07:23:45.874 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-03-13T07:23:45.900 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-03-13T07:23:45.922 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-03-13T07:23:45.939 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-03-13T07:23:46.134 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 5/34 2024-03-13T07:23:46.186 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 5/34 2024-03-13T07:23:46.551 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-03-13T07:23:46.584 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librados2-2:19.0.0-2007.gb7c579c9.el9.x86_64 7/34 2024-03-13T07:23:46.676 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:19.0.0-2007.gb7c579c9.el9.x86_64 7/34 2024-03-13T07:23:46.677 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:23:46.768 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rados-2:19.0.0-2007.gb7c579c9.el9.x86_64 8/34 2024-03-13T07:23:46.769 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-03-13T07:23:46.795 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs2-2:19.0.0-2007.gb7c579c9.el9.x86_64 9/34 2024-03-13T07:23:46.801 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-03-13T07:23:46.827 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-03-13T07:23:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-03-13T07:23:46.859 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephfs2-2:19.0.0-2007.gb7c579c9.el9.x86_64 9/34 2024-03-13T07:23:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-03-13T07:23:46.899 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-03-13T07:23:46.940 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-argparse-2:19.0.0-2007.gb7c579c9.el9. 11/34 2024-03-13T07:23:47.007 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cephfs-2:19.0.0-2007.gb7c579c9.el9.x86_64 12/34 2024-03-13T07:23:47.016 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 5/34 2024-03-13T07:23:47.030 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libradosstriper1-2:19.0.0-2007.gb7c579c9.el9.x86_6 13/34 2024-03-13T07:23:47.072 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 5/34 2024-03-13T07:23:47.093 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2007.gb7c579c9.el9.x86_6 13/34 2024-03-13T07:23:47.120 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 14/34 2024-03-13T07:23:47.229 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-03-13T07:23:47.263 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-03-13T07:23:47.304 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-03-13T07:23:47.336 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-03-13T07:23:47.365 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-03-13T07:23:47.391 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-03-13T07:23:47.406 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-03-13T07:23:47.454 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librados2-2:19.0.0-2007.gb7c579c9.el9.x86_64 7/34 2024-03-13T07:23:47.545 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:19.0.0-2007.gb7c579c9.el9.x86_64 7/34 2024-03-13T07:23:47.635 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rados-2:19.0.0-2007.gb7c579c9.el9.x86_64 8/34 2024-03-13T07:23:47.664 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs2-2:19.0.0-2007.gb7c579c9.el9.x86_64 9/34 2024-03-13T07:23:47.723 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-03-13T07:23:47.729 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephfs2-2:19.0.0-2007.gb7c579c9.el9.x86_64 9/34 2024-03-13T07:23:47.753 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librbd1-2:19.0.0-2007.gb7c579c9.el9.x86_64 22/34 2024-03-13T07:23:47.764 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-03-13T07:23:47.819 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-argparse-2:19.0.0-2007.gb7c579c9.el9. 11/34 2024-03-13T07:23:47.842 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:19.0.0-2007.gb7c579c9.el9.x86_64 22/34 2024-03-13T07:23:47.874 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rbd-2:19.0.0-2007.gb7c579c9.el9.x86_64 23/34 2024-03-13T07:23:47.906 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cephfs-2:19.0.0-2007.gb7c579c9.el9.x86_64 12/34 2024-03-13T07:23:47.926 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-03-13T07:23:47.933 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libradosstriper1-2:19.0.0-2007.gb7c579c9.el9.x86_6 13/34 2024-03-13T07:23:47.979 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2007.gb7c579c9.el9.x86_6 13/34 2024-03-13T07:23:48.015 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 14/34 2024-03-13T07:23:48.126 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-03-13T07:23:48.157 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-03-13T07:23:48.198 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-03-13T07:23:48.239 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-03-13T07:23:48.268 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-03-13T07:23:48.301 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-03-13T07:23:48.385 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-03-13T07:23:48.414 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librgw2-2:19.0.0-2007.gb7c579c9.el9.x86_64 26/34 2024-03-13T07:23:48.484 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librgw2-2:19.0.0-2007.gb7c579c9.el9.x86_64 26/34 2024-03-13T07:23:48.552 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rgw-2:19.0.0-2007.gb7c579c9.el9.x86_64 27/34 2024-03-13T07:23:48.583 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-common-2:19.0.0-2007.gb7c579c9.el9.x8 28/34 2024-03-13T07:23:48.626 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-03-13T07:23:48.656 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librbd1-2:19.0.0-2007.gb7c579c9.el9.x86_64 22/34 2024-03-13T07:23:48.746 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:19.0.0-2007.gb7c579c9.el9.x86_64 22/34 2024-03-13T07:23:48.776 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rbd-2:19.0.0-2007.gb7c579c9.el9.x86_64 23/34 2024-03-13T07:23:48.819 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-03-13T07:23:49.273 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-03-13T07:23:49.308 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librgw2-2:19.0.0-2007.gb7c579c9.el9.x86_64 26/34 2024-03-13T07:23:49.387 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librgw2-2:19.0.0-2007.gb7c579c9.el9.x86_64 26/34 2024-03-13T07:23:49.473 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rgw-2:19.0.0-2007.gb7c579c9.el9.x86_64 27/34 2024-03-13T07:23:49.502 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-common-2:19.0.0-2007.gb7c579c9.el9.x8 28/34 2024-03-13T07:23:50.702 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:19.0.0-2007.gb7c579c9.el9.x86_64 29/34 2024-03-13T07:23:50.754 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-common-2:19.0.0-2007.gb7c579c9.el9.x86_64 29/34 2024-03-13T07:23:51.303 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:19.0.0-2007.gb7c579c9.el9.x86_64 29/34 2024-03-13T07:23:51.349 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-base-2:19.0.0-2007.gb7c579c9.el9.x86_64 30/34 2024-03-13T07:23:51.440 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-base-2:19.0.0-2007.gb7c579c9.el9.x86_64 30/34 2024-03-13T07:23:51.440 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-13T07:23:51.441 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-13T07:23:51.441 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:23:51.484 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-selinux-2:19.0.0-2007.gb7c579c9.el9.x86_64 31/34 2024-03-13T07:23:51.615 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:19.0.0-2007.gb7c579c9.el9.x86_64 29/34 2024-03-13T07:23:51.665 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-common-2:19.0.0-2007.gb7c579c9.el9.x86_64 29/34 2024-03-13T07:23:52.230 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:19.0.0-2007.gb7c579c9.el9.x86_64 29/34 2024-03-13T07:23:52.268 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-base-2:19.0.0-2007.gb7c579c9.el9.x86_64 30/34 2024-03-13T07:23:52.345 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-base-2:19.0.0-2007.gb7c579c9.el9.x86_64 30/34 2024-03-13T07:23:52.345 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-13T07:23:52.345 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-13T07:23:52.345 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:23:52.387 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-selinux-2:19.0.0-2007.gb7c579c9.el9.x86_64 31/34 2024-03-13T07:24:01.398 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2007.gb7c579c9.el9.x86_64 31/34 2024-03-13T07:24:01.398 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /sys 2024-03-13T07:24:01.398 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /proc 2024-03-13T07:24:01.398 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /mnt 2024-03-13T07:24:01.398 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /var/tmp 2024-03-13T07:24:01.398 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /home 2024-03-13T07:24:01.398 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /root 2024-03-13T07:24:01.398 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /tmp 2024-03-13T07:24:01.398 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:02.154 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2007.gb7c579c9.el9.x86_64 31/34 2024-03-13T07:24:02.154 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /sys 2024-03-13T07:24:02.154 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /proc 2024-03-13T07:24:02.154 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /mnt 2024-03-13T07:24:02.154 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /var/tmp 2024-03-13T07:24:02.154 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /home 2024-03-13T07:24:02.154 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /root 2024-03-13T07:24:02.154 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /tmp 2024-03-13T07:24:02.154 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:02.315 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-radosgw-2:19.0.0-2007.gb7c579c9.el9.x86_64 32/34 2024-03-13T07:24:02.339 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2007.gb7c579c9.el9.x86_64 32/34 2024-03-13T07:24:02.340 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T07:24:02.340 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-13T07:24:02.340 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-13T07:24:02.340 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-13T07:24:02.340 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:02.341 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-13T07:24:02.409 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-13T07:24:02.409 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-13T07:24:03.083 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-radosgw-2:19.0.0-2007.gb7c579c9.el9.x86_64 32/34 2024-03-13T07:24:03.108 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2007.gb7c579c9.el9.x86_64 32/34 2024-03-13T07:24:03.108 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T07:24:03.108 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-13T07:24:03.108 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-13T07:24:03.108 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-13T07:24:03.108 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:03.110 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-13T07:24:03.178 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-13T07:24:03.178 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-13T07:24:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-13T07:24:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-base-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/34 2024-03-13T07:24:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-common-2:19.0.0-2007.gb7c579c9.el9.x86_64 2/34 2024-03-13T07:24:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-radosgw-2:19.0.0-2007.gb7c579c9.el9.x86_64 3/34 2024-03-13T07:24:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-selinux-2:19.0.0-2007.gb7c579c9.el9.x86_64 4/34 2024-03-13T07:24:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs2-2:19.0.0-2007.gb7c579c9.el9.x86_64 5/34 2024-03-13T07:24:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libradosstriper1-2:19.0.0-2007.gb7c579c9.el9.x86_6 6/34 2024-03-13T07:24:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librgw2-2:19.0.0-2007.gb7c579c9.el9.x86_64 7/34 2024-03-13T07:24:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2007.gb7c579c9.el9. 8/34 2024-03-13T07:24:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-common-2:19.0.0-2007.gb7c579c9.el9.x8 9/34 2024-03-13T07:24:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cephfs-2:19.0.0-2007.gb7c579c9.el9.x86_64 10/34 2024-03-13T07:24:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rados-2:19.0.0-2007.gb7c579c9.el9.x86_64 11/34 2024-03-13T07:24:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rbd-2:19.0.0-2007.gb7c579c9.el9.x86_64 12/34 2024-03-13T07:24:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rgw-2:19.0.0-2007.gb7c579c9.el9.x86_64 13/34 2024-03-13T07:24:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-03-13T07:24:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-03-13T07:24:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-03-13T07:24:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-03-13T07:24:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-03-13T07:24:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-03-13T07:24:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-03-13T07:24:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-03-13T07:24:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-03-13T07:24:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-03-13T07:24:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-03-13T07:24:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-03-13T07:24:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-03-13T07:24:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-03-13T07:24:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-03-13T07:24:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-03-13T07:24:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-03-13T07:24:03.623 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:19.0.0-2007.gb7c579c9.el9.x86_64 31/34 2024-03-13T07:24:03.623 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-03-13T07:24:03.623 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:19.0.0-2007.gb7c579c9.el9.x86_64 33/34 2024-03-13T07:24:04.365 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-13T07:24:04.365 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-base-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/34 2024-03-13T07:24:04.365 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-common-2:19.0.0-2007.gb7c579c9.el9.x86_64 2/34 2024-03-13T07:24:04.365 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-radosgw-2:19.0.0-2007.gb7c579c9.el9.x86_64 3/34 2024-03-13T07:24:04.365 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-selinux-2:19.0.0-2007.gb7c579c9.el9.x86_64 4/34 2024-03-13T07:24:04.365 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs2-2:19.0.0-2007.gb7c579c9.el9.x86_64 5/34 2024-03-13T07:24:04.365 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libradosstriper1-2:19.0.0-2007.gb7c579c9.el9.x86_6 6/34 2024-03-13T07:24:04.365 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librgw2-2:19.0.0-2007.gb7c579c9.el9.x86_64 7/34 2024-03-13T07:24:04.365 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2007.gb7c579c9.el9. 8/34 2024-03-13T07:24:04.365 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-common-2:19.0.0-2007.gb7c579c9.el9.x8 9/34 2024-03-13T07:24:04.365 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cephfs-2:19.0.0-2007.gb7c579c9.el9.x86_64 10/34 2024-03-13T07:24:04.365 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rados-2:19.0.0-2007.gb7c579c9.el9.x86_64 11/34 2024-03-13T07:24:04.366 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rbd-2:19.0.0-2007.gb7c579c9.el9.x86_64 12/34 2024-03-13T07:24:04.366 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rgw-2:19.0.0-2007.gb7c579c9.el9.x86_64 13/34 2024-03-13T07:24:04.366 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-03-13T07:24:04.366 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-03-13T07:24:04.366 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-03-13T07:24:04.366 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-03-13T07:24:04.366 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-03-13T07:24:04.366 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-03-13T07:24:04.366 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-03-13T07:24:04.366 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-03-13T07:24:04.366 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-03-13T07:24:04.366 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-03-13T07:24:04.366 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-03-13T07:24:04.366 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-03-13T07:24:04.367 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-03-13T07:24:04.367 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-03-13T07:24:04.367 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-03-13T07:24:04.368 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-03-13T07:24:04.368 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-03-13T07:24:04.368 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:19.0.0-2007.gb7c579c9.el9.x86_64 31/34 2024-03-13T07:24:04.368 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-03-13T07:24:04.368 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:19.0.0-2007.gb7c579c9.el9.x86_64 33/34 2024-03-13T07:24:04.727 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-03-13T07:24:04.727 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:04.727 INFO:teuthology.orchestra.run.smithi132.stdout:Upgraded: 2024-03-13T07:24:04.727 INFO:teuthology.orchestra.run.smithi132.stdout: librados2-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:04.728 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:04.728 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:24:04.728 INFO:teuthology.orchestra.run.smithi132.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-13T07:24:04.728 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:04.728 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:04.728 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:04.728 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:04.728 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-13T07:24:04.728 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-13T07:24:04.728 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-13T07:24:04.728 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:04.728 INFO:teuthology.orchestra.run.smithi132.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-13T07:24:04.728 INFO:teuthology.orchestra.run.smithi132.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-13T07:24:04.728 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-13T07:24:04.729 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:04.729 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-13T07:24:04.729 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:04.729 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-13T07:24:04.729 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-13T07:24:04.729 INFO:teuthology.orchestra.run.smithi132.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-13T07:24:04.729 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:04.729 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:04.729 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:04.729 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-13T07:24:04.729 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:04.729 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:04.729 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:04.729 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-13T07:24:04.729 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-13T07:24:04.730 INFO:teuthology.orchestra.run.smithi132.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-13T07:24:04.730 INFO:teuthology.orchestra.run.smithi132.stdout: re2-1:20211101-3.el9.x86_64 2024-03-13T07:24:04.730 INFO:teuthology.orchestra.run.smithi132.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-13T07:24:04.730 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:04.730 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:24:05.130 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-test 2024-03-13T07:24:05.368 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-03-13T07:24:05.369 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:05.369 INFO:teuthology.orchestra.run.smithi195.stdout:Upgraded: 2024-03-13T07:24:05.369 INFO:teuthology.orchestra.run.smithi195.stdout: librados2-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:05.369 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:05.369 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:24:05.369 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-13T07:24:05.369 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:05.369 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:05.369 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:05.369 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:05.369 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-13T07:24:05.369 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-13T07:24:05.369 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-13T07:24:05.369 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:05.370 INFO:teuthology.orchestra.run.smithi195.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-13T07:24:05.370 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-13T07:24:05.370 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-13T07:24:05.370 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:05.370 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-13T07:24:05.370 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:05.370 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-13T07:24:05.370 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-13T07:24:05.370 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-13T07:24:05.370 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:05.370 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:05.370 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:05.371 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-13T07:24:05.371 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:05.371 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:05.371 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:05.371 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-13T07:24:05.371 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-13T07:24:05.371 INFO:teuthology.orchestra.run.smithi195.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-13T07:24:05.371 INFO:teuthology.orchestra.run.smithi195.stdout: re2-1:20211101-3.el9.x86_64 2024-03-13T07:24:05.371 INFO:teuthology.orchestra.run.smithi195.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-13T07:24:05.371 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:05.371 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:24:05.690 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:29 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:24:05.752 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-test 2024-03-13T07:24:05.842 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:24:05.843 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:24:05.843 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-03-13T07:24:05.843 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:24:05.843 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:24:05.843 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 48 M 2024-03-13T07:24:05.843 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-03-13T07:24:05.843 INFO:teuthology.orchestra.run.smithi132.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-13T07:24:05.843 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 163 k 2024-03-13T07:24:05.843 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-13T07:24:05.843 INFO:teuthology.orchestra.run.smithi132.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-13T07:24:05.843 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-13T07:24:05.843 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:05.843 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:24:05.844 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:24:05.844 INFO:teuthology.orchestra.run.smithi132.stdout:Install 6 Packages 2024-03-13T07:24:05.844 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:05.844 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 48 M 2024-03-13T07:24:05.844 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 205 M 2024-03-13T07:24:05.844 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:24:06.303 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:00:29 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:24:06.406 INFO:teuthology.orchestra.run.smithi132.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.0 MB/s | 217 kB 00:00 2024-03-13T07:24:06.456 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:24:06.457 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:24:06.457 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-03-13T07:24:06.457 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:24:06.457 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:24:06.457 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 48 M 2024-03-13T07:24:06.457 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-03-13T07:24:06.458 INFO:teuthology.orchestra.run.smithi195.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-13T07:24:06.458 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 163 k 2024-03-13T07:24:06.458 INFO:teuthology.orchestra.run.smithi195.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-13T07:24:06.458 INFO:teuthology.orchestra.run.smithi195.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-13T07:24:06.458 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-13T07:24:06.458 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:06.458 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:24:06.458 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:24:06.458 INFO:teuthology.orchestra.run.smithi195.stdout:Install 6 Packages 2024-03-13T07:24:06.458 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:06.459 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 48 M 2024-03-13T07:24:06.459 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 205 M 2024-03-13T07:24:06.459 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:24:06.482 INFO:teuthology.orchestra.run.smithi132.stdout:(2/6): libcephsqlite-19.0.0-2007.gb7c579c9.el9. 574 kB/s | 163 kB 00:00 2024-03-13T07:24:06.532 INFO:teuthology.orchestra.run.smithi132.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 2.3 MB/s | 300 kB 00:00 2024-03-13T07:24:06.560 INFO:teuthology.orchestra.run.smithi132.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 809 kB/s | 63 kB 00:00 2024-03-13T07:24:06.692 INFO:teuthology.orchestra.run.smithi195.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.1 MB/s | 217 kB 00:00 2024-03-13T07:24:06.693 INFO:teuthology.orchestra.run.smithi132.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-03-13T07:24:06.751 INFO:teuthology.orchestra.run.smithi195.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-03-13T07:24:06.784 INFO:teuthology.orchestra.run.smithi195.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-13T07:24:06.910 INFO:teuthology.orchestra.run.smithi195.stdout:(4/6): libcephsqlite-19.0.0-2007.gb7c579c9.el9. 397 kB/s | 163 kB 00:00 2024-03-13T07:24:07.301 INFO:teuthology.orchestra.run.smithi195.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 362 kB/s | 187 kB 00:00 2024-03-13T07:24:08.626 INFO:teuthology.orchestra.run.smithi132.stdout:(6/6): ceph-test-19.0.0-2007.gb7c579c9.el9.x86_ 20 MB/s | 48 MB 00:02 2024-03-13T07:24:08.629 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:24:08.630 INFO:teuthology.orchestra.run.smithi132.stdout:Total 17 MB/s | 48 MB 00:02 2024-03-13T07:24:08.644 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:24:08.666 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:24:08.666 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:24:09.054 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:24:09.055 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:24:09.637 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:24:09.702 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-13T07:24:09.743 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-13T07:24:09.776 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-13T07:24:09.819 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-13T07:24:09.846 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephsqlite-2:19.0.0-2007.gb7c579c9.el9.x86_64 5/6 2024-03-13T07:24:13.643 INFO:teuthology.orchestra.run.smithi195.stdout:(6/6): ceph-test-19.0.0-2007.gb7c579c9.el9.x86_ 6.7 MB/s | 48 MB 00:07 2024-03-13T07:24:13.646 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:24:13.646 INFO:teuthology.orchestra.run.smithi195.stdout:Total 6.7 MB/s | 48 MB 00:07 2024-03-13T07:24:13.660 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:24:13.681 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:24:13.681 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:24:14.064 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:24:14.064 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:24:14.368 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2007.gb7c579c9.el9.x86_64 5/6 2024-03-13T07:24:14.446 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-test-2:19.0.0-2007.gb7c579c9.el9.x86_64 6/6 2024-03-13T07:24:14.634 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:24:14.702 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-13T07:24:14.742 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-13T07:24:14.775 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-13T07:24:14.822 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-13T07:24:14.846 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephsqlite-2:19.0.0-2007.gb7c579c9.el9.x86_64 5/6 2024-03-13T07:24:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-test-2:19.0.0-2007.gb7c579c9.el9.x86_64 6/6 2024-03-13T07:24:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-test-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/6 2024-03-13T07:24:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephsqlite-2:19.0.0-2007.gb7c579c9.el9.x86_64 2/6 2024-03-13T07:24:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-13T07:24:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-13T07:24:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-13T07:24:17.259 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-13T07:24:17.259 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:17.259 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:24:17.259 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:17.259 INFO:teuthology.orchestra.run.smithi132.stdout: jq-1.6-16.el9.x86_64 2024-03-13T07:24:17.259 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:17.259 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-13T07:24:17.259 INFO:teuthology.orchestra.run.smithi132.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-13T07:24:17.260 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-13T07:24:17.260 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:17.260 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:24:17.589 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph 2024-03-13T07:24:18.190 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:42 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:24:18.350 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:24:18.353 INFO:teuthology.orchestra.run.smithi132.stdout:========================================================================================= 2024-03-13T07:24:18.353 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-03-13T07:24:18.353 INFO:teuthology.orchestra.run.smithi132.stdout:========================================================================================= 2024-03-13T07:24:18.353 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:24:18.353 INFO:teuthology.orchestra.run.smithi132.stdout: ceph x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 6.6 k 2024-03-13T07:24:18.353 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-03-13T07:24:18.354 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 2.1 M 2024-03-13T07:24:18.354 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 1.5 M 2024-03-13T07:24:18.354 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 248 k 2024-03-13T07:24:18.354 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 4.7 M 2024-03-13T07:24:18.354 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 17 M 2024-03-13T07:24:18.354 INFO:teuthology.orchestra.run.smithi132.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-13T07:24:18.354 INFO:teuthology.orchestra.run.smithi132.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-13T07:24:18.354 INFO:teuthology.orchestra.run.smithi132.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-13T07:24:18.354 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-13T07:24:18.354 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-13T07:24:18.354 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-13T07:24:18.354 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-13T07:24:18.355 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-13T07:24:18.355 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-13T07:24:18.355 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-13T07:24:18.355 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-13T07:24:18.355 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-13T07:24:18.355 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-13T07:24:18.355 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-13T07:24:18.355 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-13T07:24:18.355 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-13T07:24:18.355 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-13T07:24:18.355 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-13T07:24:18.355 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-13T07:24:18.355 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-13T07:24:18.356 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-13T07:24:18.356 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-13T07:24:18.356 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-13T07:24:18.356 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-13T07:24:18.356 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-13T07:24:18.356 INFO:teuthology.orchestra.run.smithi132.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-13T07:24:18.356 INFO:teuthology.orchestra.run.smithi132.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-13T07:24:18.356 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-13T07:24:18.356 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-13T07:24:18.356 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-13T07:24:18.356 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:18.356 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:24:18.356 INFO:teuthology.orchestra.run.smithi132.stdout:========================================================================================= 2024-03-13T07:24:18.357 INFO:teuthology.orchestra.run.smithi132.stdout:Install 36 Packages 2024-03-13T07:24:18.357 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:18.357 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 30 M 2024-03-13T07:24:18.357 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 107 M 2024-03-13T07:24:18.357 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:24:19.072 INFO:teuthology.orchestra.run.smithi132.stdout:(1/36): ceph-19.0.0-2007.gb7c579c9.el9.x86_64.r 38 kB/s | 6.6 kB 00:00 2024-03-13T07:24:19.272 INFO:teuthology.orchestra.run.smithi132.stdout:(2/36): ceph-mds-19.0.0-2007.gb7c579c9.el9.x86_ 5.7 MB/s | 2.1 MB 00:00 2024-03-13T07:24:19.378 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2007.gb7c579c9.el9.x86_64 5/6 2024-03-13T07:24:19.455 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-test-2:19.0.0-2007.gb7c579c9.el9.x86_64 6/6 2024-03-13T07:24:19.589 INFO:teuthology.orchestra.run.smithi132.stdout:(3/36): ceph-mgr-19.0.0-2007.gb7c579c9.el9.x86_ 2.2 MB/s | 1.5 MB 00:00 2024-03-13T07:24:19.714 INFO:teuthology.orchestra.run.smithi132.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2007.gb7c5 1.9 MB/s | 248 kB 00:00 2024-03-13T07:24:19.856 INFO:teuthology.orchestra.run.smithi132.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2024-03-13T07:24:19.923 INFO:teuthology.orchestra.run.smithi132.stdout:(6/36): ceph-mon-19.0.0-2007.gb7c579c9.el9.x86_ 5.5 MB/s | 4.7 MB 00:00 2024-03-13T07:24:19.948 INFO:teuthology.orchestra.run.smithi132.stdout:(7/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 144 kB 00:00 2024-03-13T07:24:19.966 INFO:teuthology.orchestra.run.smithi132.stdout:(8/36): python3-markupsafe-1.1.1-12.el9.x86_64. 758 kB/s | 32 kB 00:00 2024-03-13T07:24:19.999 INFO:teuthology.orchestra.run.smithi132.stdout:(9/36): python3-pytz-2021.1-5.el9.noarch.rpm 944 kB/s | 47 kB 00:00 2024-03-13T07:24:20.015 INFO:teuthology.orchestra.run.smithi132.stdout:(10/36): python3-toml-0.10.2-6.el9.noarch.rpm 772 kB/s | 39 kB 00:00 2024-03-13T07:24:20.041 INFO:teuthology.orchestra.run.smithi132.stdout:(11/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-03-13T07:24:20.100 INFO:teuthology.orchestra.run.smithi132.stdout:(12/36): python3-cryptography-36.0.1-4.el9.x86_ 14 MB/s | 1.1 MB 00:00 2024-03-13T07:24:20.125 INFO:teuthology.orchestra.run.smithi132.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-13T07:24:20.142 INFO:teuthology.orchestra.run.smithi132.stdout:(14/36): python3-pycparser-2.20-6.el9.noarch.rp 2.9 MB/s | 124 kB 00:00 2024-03-13T07:24:20.170 INFO:teuthology.orchestra.run.smithi132.stdout:(15/36): python3-urllib3-1.26.5-5.el9.noarch.rp 4.4 MB/s | 187 kB 00:00 2024-03-13T07:24:20.335 INFO:teuthology.orchestra.run.smithi132.stdout:(16/36): ceph-osd-19.0.0-2007.gb7c579c9.el9.x86 16 MB/s | 17 MB 00:01 2024-03-13T07:24:20.368 INFO:teuthology.orchestra.run.smithi132.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 111 kB/s | 22 kB 00:00 2024-03-13T07:24:20.426 INFO:teuthology.orchestra.run.smithi132.stdout:(18/36): python3-requests-2.25.1-8.el9.noarch.r 440 kB/s | 125 kB 00:00 2024-03-13T07:24:20.518 INFO:teuthology.orchestra.run.smithi132.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 289 kB/s | 43 kB 00:00 2024-03-13T07:24:20.544 INFO:teuthology.orchestra.run.smithi132.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 724 kB/s | 151 kB 00:00 2024-03-13T07:24:20.602 INFO:teuthology.orchestra.run.smithi132.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 183 kB/s | 11 kB 00:00 2024-03-13T07:24:20.628 INFO:teuthology.orchestra.run.smithi132.stdout:(22/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 855 kB/s | 172 kB 00:00 2024-03-13T07:24:20.661 INFO:teuthology.orchestra.run.smithi132.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 302 kB/s | 18 kB 00:00 2024-03-13T07:24:20.686 INFO:teuthology.orchestra.run.smithi132.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 394 kB/s | 23 kB 00:00 2024-03-13T07:24:20.712 INFO:teuthology.orchestra.run.smithi132.stdout:(25/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.8 MB/s | 358 kB 00:00 2024-03-13T07:24:20.737 INFO:teuthology.orchestra.run.smithi132.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 257 kB/s | 19 kB 00:00 2024-03-13T07:24:20.762 INFO:teuthology.orchestra.run.smithi132.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 260 kB/s | 20 kB 00:00 2024-03-13T07:24:20.788 INFO:teuthology.orchestra.run.smithi132.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 614 kB/s | 46 kB 00:00 2024-03-13T07:24:20.813 INFO:teuthology.orchestra.run.smithi132.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-03-13T07:24:20.847 INFO:teuthology.orchestra.run.smithi132.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 280 kB/s | 16 kB 00:00 2024-03-13T07:24:20.872 INFO:teuthology.orchestra.run.smithi132.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-03-13T07:24:20.897 INFO:teuthology.orchestra.run.smithi132.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.0 MB/s | 90 kB 00:00 2024-03-13T07:24:20.922 INFO:teuthology.orchestra.run.smithi132.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 475 kB/s | 36 kB 00:00 2024-03-13T07:24:20.948 INFO:teuthology.orchestra.run.smithi132.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-13T07:24:20.973 INFO:teuthology.orchestra.run.smithi132.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 399 kB/s | 20 kB 00:00 2024-03-13T07:24:21.023 INFO:teuthology.orchestra.run.smithi132.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2024-03-13T07:24:21.027 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:24:21.027 INFO:teuthology.orchestra.run.smithi132.stdout:Total 11 MB/s | 30 MB 00:02 2024-03-13T07:24:21.051 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:24:21.092 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:24:21.092 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:24:21.506 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:24:21.506 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:24:21.869 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-test-2:19.0.0-2007.gb7c579c9.el9.x86_64 6/6 2024-03-13T07:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-test-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/6 2024-03-13T07:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephsqlite-2:19.0.0-2007.gb7c579c9.el9.x86_64 2/6 2024-03-13T07:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-13T07:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-13T07:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-13T07:24:22.283 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-13T07:24:22.283 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:22.283 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:24:22.283 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:22.283 INFO:teuthology.orchestra.run.smithi195.stdout: jq-1.6-16.el9.x86_64 2024-03-13T07:24:22.283 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:22.283 INFO:teuthology.orchestra.run.smithi195.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-13T07:24:22.283 INFO:teuthology.orchestra.run.smithi195.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-13T07:24:22.283 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-13T07:24:22.283 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:22.283 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:24:22.598 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph 2024-03-13T07:24:22.885 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:24:22.942 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-03-13T07:24:22.988 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-03-13T07:24:23.020 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-03-13T07:24:23.092 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-03-13T07:24:23.144 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-03-13T07:24:23.172 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:00:46 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:24:23.177 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-03-13T07:24:23.268 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-03-13T07:24:23.323 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:24:23.325 INFO:teuthology.orchestra.run.smithi195.stdout:========================================================================================= 2024-03-13T07:24:23.325 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-03-13T07:24:23.325 INFO:teuthology.orchestra.run.smithi195.stdout:========================================================================================= 2024-03-13T07:24:23.325 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:24:23.326 INFO:teuthology.orchestra.run.smithi195.stdout: ceph x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 6.6 k 2024-03-13T07:24:23.326 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-03-13T07:24:23.326 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 2.1 M 2024-03-13T07:24:23.326 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 1.5 M 2024-03-13T07:24:23.326 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 248 k 2024-03-13T07:24:23.326 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 4.7 M 2024-03-13T07:24:23.326 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 17 M 2024-03-13T07:24:23.326 INFO:teuthology.orchestra.run.smithi195.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-13T07:24:23.326 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-13T07:24:23.326 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-13T07:24:23.326 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-13T07:24:23.326 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-13T07:24:23.326 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-13T07:24:23.327 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-13T07:24:23.327 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-13T07:24:23.327 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-13T07:24:23.327 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-13T07:24:23.327 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-13T07:24:23.327 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-13T07:24:23.327 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-13T07:24:23.327 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-13T07:24:23.327 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-13T07:24:23.327 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-13T07:24:23.327 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-13T07:24:23.327 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-13T07:24:23.327 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-13T07:24:23.327 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-13T07:24:23.328 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-13T07:24:23.328 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-13T07:24:23.328 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-13T07:24:23.328 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-13T07:24:23.328 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-13T07:24:23.328 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-13T07:24:23.328 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-13T07:24:23.328 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-13T07:24:23.328 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-13T07:24:23.328 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-13T07:24:23.328 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:23.328 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:24:23.328 INFO:teuthology.orchestra.run.smithi195.stdout:========================================================================================= 2024-03-13T07:24:23.328 INFO:teuthology.orchestra.run.smithi195.stdout:Install 36 Packages 2024-03-13T07:24:23.329 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:23.329 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 30 M 2024-03-13T07:24:23.329 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 107 M 2024-03-13T07:24:23.329 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:24:23.346 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-03-13T07:24:23.396 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-03-13T07:24:23.467 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-03-13T07:24:23.522 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-03-13T07:24:23.560 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-03-13T07:24:23.617 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-03-13T07:24:23.673 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-03-13T07:24:23.865 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-03-13T07:24:23.914 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-03-13T07:24:23.945 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-03-13T07:24:23.979 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-03-13T07:24:24.009 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-03-13T07:24:24.086 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-03-13T07:24:24.120 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-13T07:24:24.153 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-03-13T07:24:24.193 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-03-13T07:24:24.196 INFO:teuthology.orchestra.run.smithi195.stdout:(1/36): ceph-19.0.0-2007.gb7c579c9.el9.x86_64.r 39 kB/s | 6.6 kB 00:00 2024-03-13T07:24:24.228 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-03-13T07:24:24.272 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-03-13T07:24:24.360 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-03-13T07:24:24.403 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-03-13T07:24:24.475 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-03-13T07:24:24.596 INFO:teuthology.orchestra.run.smithi195.stdout:(2/36): ceph-mon-19.0.0-2007.gb7c579c9.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-03-13T07:24:24.612 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-03-13T07:24:24.682 INFO:teuthology.orchestra.run.smithi195.stdout:(3/36): ceph-mgr-19.0.0-2007.gb7c579c9.el9.x86_ 2.3 MB/s | 1.5 MB 00:00 2024-03-13T07:24:24.736 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-13T07:24:24.782 INFO:teuthology.orchestra.run.smithi195.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2007.gb7c5 2.4 MB/s | 248 kB 00:00 2024-03-13T07:24:24.824 INFO:teuthology.orchestra.run.smithi195.stdout:(5/36): ceph-mds-19.0.0-2007.gb7c579c9.el9.x86_ 2.7 MB/s | 2.1 MB 00:00 2024-03-13T07:24:24.904 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2007.gb7c579c9.el9. 31/36 2024-03-13T07:24:24.932 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-2:19.0.0-2007.gb7c579c9.el9.x86_64 32/36 2024-03-13T07:24:24.949 INFO:teuthology.orchestra.run.smithi195.stdout:(6/36): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-03-13T07:24:24.958 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2007.gb7c579c9.el9.x86_64 32/36 2024-03-13T07:24:24.958 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T07:24:24.958 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-13T07:24:24.958 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-13T07:24:24.958 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-13T07:24:24.958 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:25.000 INFO:teuthology.orchestra.run.smithi195.stdout:(7/36): python3-mako-1.1.4-6.el9.noarch.rpm 823 kB/s | 144 kB 00:00 2024-03-13T07:24:25.025 INFO:teuthology.orchestra.run.smithi195.stdout:(8/36): python3-markupsafe-1.1.1-12.el9.x86_64. 421 kB/s | 32 kB 00:00 2024-03-13T07:24:25.050 INFO:teuthology.orchestra.run.smithi195.stdout:(9/36): python3-pytz-2021.1-5.el9.noarch.rpm 939 kB/s | 47 kB 00:00 2024-03-13T07:24:25.076 INFO:teuthology.orchestra.run.smithi195.stdout:(10/36): python3-toml-0.10.2-6.el9.noarch.rpm 767 kB/s | 39 kB 00:00 2024-03-13T07:24:25.101 INFO:teuthology.orchestra.run.smithi195.stdout:(11/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-13T07:24:25.159 INFO:teuthology.orchestra.run.smithi195.stdout:(12/36): python3-cryptography-36.0.1-4.el9.x86_ 14 MB/s | 1.1 MB 00:00 2024-03-13T07:24:25.185 INFO:teuthology.orchestra.run.smithi195.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-13T07:24:25.210 INFO:teuthology.orchestra.run.smithi195.stdout:(14/36): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-03-13T07:24:25.235 INFO:teuthology.orchestra.run.smithi195.stdout:(15/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-03-13T07:24:25.444 INFO:teuthology.orchestra.run.smithi195.stdout:(16/36): ceph-osd-19.0.0-2007.gb7c579c9.el9.x86 20 MB/s | 17 MB 00:00 2024-03-13T07:24:25.486 INFO:teuthology.orchestra.run.smithi195.stdout:(17/36): python3-requests-2.25.1-8.el9.noarch.r 454 kB/s | 125 kB 00:00 2024-03-13T07:24:25.661 INFO:teuthology.orchestra.run.smithi195.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 52 kB/s | 22 kB 00:00 2024-03-13T07:24:25.828 INFO:teuthology.orchestra.run.smithi195.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 127 kB/s | 43 kB 00:00 2024-03-13T07:24:25.870 INFO:teuthology.orchestra.run.smithi195.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 355 kB/s | 151 kB 00:00 2024-03-13T07:24:26.045 INFO:teuthology.orchestra.run.smithi195.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 61 kB/s | 11 kB 00:00 2024-03-13T07:24:26.095 INFO:teuthology.orchestra.run.smithi195.stdout:(22/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 396 kB/s | 172 kB 00:00 2024-03-13T07:24:26.304 INFO:teuthology.orchestra.run.smithi195.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 68 kB/s | 18 kB 00:00 2024-03-13T07:24:26.354 INFO:teuthology.orchestra.run.smithi195.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 89 kB/s | 23 kB 00:00 2024-03-13T07:24:26.408 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-osd-2:19.0.0-2007.gb7c579c9.el9.x86_64 33/36 2024-03-13T07:24:26.413 INFO:teuthology.orchestra.run.smithi195.stdout:(25/36): python3-cherrypy-18.6.1-2.el9.noarch.r 613 kB/s | 358 kB 00:00 2024-03-13T07:24:26.434 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-osd-2:19.0.0-2007.gb7c579c9.el9.x86_64 33/36 2024-03-13T07:24:26.434 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T07:24:26.434 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-13T07:24:26.434 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-13T07:24:26.435 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-13T07:24:26.435 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:26.563 INFO:teuthology.orchestra.run.smithi195.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 75 kB/s | 19 kB 00:00 2024-03-13T07:24:26.622 INFO:teuthology.orchestra.run.smithi195.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 74 kB/s | 20 kB 00:00 2024-03-13T07:24:26.755 INFO:teuthology.orchestra.run.smithi195.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 135 kB/s | 46 kB 00:00 2024-03-13T07:24:26.901 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mon-2:19.0.0-2007.gb7c579c9.el9.x86_64 34/36 2024-03-13T07:24:26.905 INFO:teuthology.orchestra.run.smithi195.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 230 kB/s | 79 kB 00:00 2024-03-13T07:24:26.924 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mon-2:19.0.0-2007.gb7c579c9.el9.x86_64 34/36 2024-03-13T07:24:26.924 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T07:24:26.924 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-13T07:24:26.924 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-13T07:24:26.925 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-13T07:24:26.925 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:27.022 INFO:teuthology.orchestra.run.smithi195.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 61 kB/s | 16 kB 00:00 2024-03-13T07:24:27.131 INFO:teuthology.orchestra.run.smithi195.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 535 kB/s | 272 kB 00:00 2024-03-13T07:24:27.134 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mds-2:19.0.0-2007.gb7c579c9.el9.x86_64 35/36 2024-03-13T07:24:27.158 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mds-2:19.0.0-2007.gb7c579c9.el9.x86_64 35/36 2024-03-13T07:24:27.158 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T07:24:27.158 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-13T07:24:27.158 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-13T07:24:27.158 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-13T07:24:27.158 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:27.224 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-2:19.0.0-2007.gb7c579c9.el9.x86_64 36/36 2024-03-13T07:24:27.248 INFO:teuthology.orchestra.run.smithi195.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 263 kB/s | 90 kB 00:00 2024-03-13T07:24:27.290 INFO:teuthology.orchestra.run.smithi195.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 134 kB/s | 36 kB 00:00 2024-03-13T07:24:27.557 INFO:teuthology.orchestra.run.smithi195.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 75 kB/s | 20 kB 00:00 2024-03-13T07:24:27.641 INFO:teuthology.orchestra.run.smithi195.stdout:(35/36): python3-webob-1.8.7-6.el9.noarch.rpm 451 kB/s | 230 kB 00:00 2024-03-13T07:24:27.833 INFO:teuthology.orchestra.run.smithi195.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 731 kB/s | 427 kB 00:00 2024-03-13T07:24:27.838 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:24:27.838 INFO:teuthology.orchestra.run.smithi195.stdout:Total 6.6 MB/s | 30 MB 00:04 2024-03-13T07:24:27.863 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:24:27.905 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:24:27.905 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:24:28.321 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:24:28.322 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:24:29.014 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-2:19.0.0-2007.gb7c579c9.el9.x86_64 36/36 2024-03-13T07:24:29.014 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/36 2024-03-13T07:24:29.014 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mds-2:19.0.0-2007.gb7c579c9.el9.x86_64 2/36 2024-03-13T07:24:29.014 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-2:19.0.0-2007.gb7c579c9.el9.x86_64 3/36 2024-03-13T07:24:29.014 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mon-2:19.0.0-2007.gb7c579c9.el9.x86_64 4/36 2024-03-13T07:24:29.014 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-osd-2:19.0.0-2007.gb7c579c9.el9.x86_64 5/36 2024-03-13T07:24:29.015 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2007.gb7c579c9.el9. 6/36 2024-03-13T07:24:29.015 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-03-13T07:24:29.015 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-03-13T07:24:29.015 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-03-13T07:24:29.015 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-03-13T07:24:29.015 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-03-13T07:24:29.015 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-03-13T07:24:29.015 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-03-13T07:24:29.015 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-03-13T07:24:29.015 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-03-13T07:24:29.015 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/36 2024-03-13T07:24:29.015 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/36 2024-03-13T07:24:29.015 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-03-13T07:24:29.016 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-03-13T07:24:29.016 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-03-13T07:24:29.016 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-13T07:24:29.016 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-03-13T07:24:29.016 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-03-13T07:24:29.016 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-03-13T07:24:29.016 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-03-13T07:24:29.018 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-03-13T07:24:29.018 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-03-13T07:24:29.018 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-03-13T07:24:29.018 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-03-13T07:24:29.018 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-13T07:24:29.018 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-03-13T07:24:29.018 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-03-13T07:24:29.018 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-03-13T07:24:29.018 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-03-13T07:24:29.018 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-03-13T07:24:29.701 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:24:29.760 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-03-13T07:24:29.799 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-03-13T07:24:29.829 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-03-13T07:24:29.899 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-03-13T07:24:29.944 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-03-13T07:24:29.977 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-03-13T07:24:30.004 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-03-13T07:24:30.004 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:30.004 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:24:30.004 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:30.004 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:30.004 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:30.005 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:24:30.005 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:30.005 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:30.005 INFO:teuthology.orchestra.run.smithi132.stdout: lua-5.4.4-4.el9.x86_64 2024-03-13T07:24:30.005 INFO:teuthology.orchestra.run.smithi132.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-13T07:24:30.005 INFO:teuthology.orchestra.run.smithi132.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-13T07:24:30.005 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-13T07:24:30.005 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-13T07:24:30.005 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-13T07:24:30.005 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-13T07:24:30.005 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-13T07:24:30.005 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-13T07:24:30.005 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-13T07:24:30.005 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-13T07:24:30.006 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-13T07:24:30.006 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-13T07:24:30.006 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-13T07:24:30.006 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-13T07:24:30.006 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-13T07:24:30.006 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-13T07:24:30.006 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-13T07:24:30.006 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ply-3.11-14.el9.noarch 2024-03-13T07:24:30.006 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-13T07:24:30.006 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-13T07:24:30.006 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-13T07:24:30.006 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-13T07:24:30.006 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-13T07:24:30.007 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-13T07:24:30.007 INFO:teuthology.orchestra.run.smithi132.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-13T07:24:30.007 INFO:teuthology.orchestra.run.smithi132.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-13T07:24:30.007 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-13T07:24:30.007 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-13T07:24:30.007 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-13T07:24:30.007 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:30.007 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:24:30.068 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-03-13T07:24:30.139 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-03-13T07:24:30.180 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-03-13T07:24:30.251 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-03-13T07:24:30.306 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-03-13T07:24:30.344 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-03-13T07:24:30.347 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-base 2024-03-13T07:24:30.393 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-03-13T07:24:30.448 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-03-13T07:24:30.641 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-03-13T07:24:30.690 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-03-13T07:24:30.729 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-03-13T07:24:30.911 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:54 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:24:30.947 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-03-13T07:24:30.976 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-03-13T07:24:31.007 INFO:teuthology.orchestra.run.smithi132.stdout:Package ceph-base-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:24:31.053 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-03-13T07:24:31.058 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:24:31.059 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-03-13T07:24:31.060 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:24:31.087 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-13T07:24:31.120 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-03-13T07:24:31.127 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install cephadm 2024-03-13T07:24:31.160 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-03-13T07:24:31.196 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-03-13T07:24:31.247 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-03-13T07:24:31.344 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-03-13T07:24:31.378 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-03-13T07:24:31.450 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-03-13T07:24:31.586 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-03-13T07:24:31.691 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:55 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:24:31.712 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-13T07:24:31.841 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:24:31.842 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:24:31.842 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-03-13T07:24:31.842 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:24:31.842 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:24:31.842 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 748 k 2024-03-13T07:24:31.842 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:31.842 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:24:31.842 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:24:31.842 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:24:31.842 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:31.842 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 748 k 2024-03-13T07:24:31.843 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 755 k 2024-03-13T07:24:31.843 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:24:31.871 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2007.gb7c579c9.el9. 31/36 2024-03-13T07:24:31.908 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-2:19.0.0-2007.gb7c579c9.el9.x86_64 32/36 2024-03-13T07:24:31.933 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2007.gb7c579c9.el9.x86_64 32/36 2024-03-13T07:24:31.934 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T07:24:31.934 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-13T07:24:31.934 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-13T07:24:31.934 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-13T07:24:31.934 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:32.268 INFO:teuthology.orchestra.run.smithi132.stdout:cephadm-19.0.0-2007.gb7c579c9.el9.noarch.rpm 1.7 MB/s | 748 kB 00:00 2024-03-13T07:24:32.268 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:24:32.269 INFO:teuthology.orchestra.run.smithi132.stdout:Total 1.7 MB/s | 748 kB 00:00 2024-03-13T07:24:32.269 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:24:32.274 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:24:32.274 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:24:32.279 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:24:32.280 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:24:32.355 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:24:32.770 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 1/1 2024-03-13T07:24:32.852 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 1/1 2024-03-13T07:24:33.270 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 1/1 2024-03-13T07:24:33.384 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-osd-2:19.0.0-2007.gb7c579c9.el9.x86_64 33/36 2024-03-13T07:24:33.411 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-osd-2:19.0.0-2007.gb7c579c9.el9.x86_64 33/36 2024-03-13T07:24:33.411 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T07:24:33.411 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-13T07:24:33.411 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-13T07:24:33.411 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-13T07:24:33.411 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:33.600 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 1/1 2024-03-13T07:24:33.600 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:33.601 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:24:33.601 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:24:33.601 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:33.601 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:24:33.796 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-immutable-object-cache 2024-03-13T07:24:33.860 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mon-2:19.0.0-2007.gb7c579c9.el9.x86_64 34/36 2024-03-13T07:24:33.883 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mon-2:19.0.0-2007.gb7c579c9.el9.x86_64 34/36 2024-03-13T07:24:33.884 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T07:24:33.884 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-13T07:24:33.884 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-13T07:24:33.884 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-13T07:24:33.884 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:34.094 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mds-2:19.0.0-2007.gb7c579c9.el9.x86_64 35/36 2024-03-13T07:24:34.117 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mds-2:19.0.0-2007.gb7c579c9.el9.x86_64 35/36 2024-03-13T07:24:34.117 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T07:24:34.117 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-13T07:24:34.117 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-13T07:24:34.117 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-13T07:24:34.118 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:34.184 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-2:19.0.0-2007.gb7c579c9.el9.x86_64 36/36 2024-03-13T07:24:34.366 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:58 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:24:34.521 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:24:34.522 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:24:34.522 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:24:34.522 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:24:34.522 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:24:34.522 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 145 k 2024-03-13T07:24:34.522 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:34.522 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:24:34.522 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:24:34.522 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:24:34.523 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:34.523 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 145 k 2024-03-13T07:24:34.523 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 436 k 2024-03-13T07:24:34.523 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:24:34.869 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-immutable-object-cache-19.0.0-2007.gb7c579 419 kB/s | 145 kB 00:00 2024-03-13T07:24:34.869 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:24:34.869 INFO:teuthology.orchestra.run.smithi132.stdout:Total 418 kB/s | 145 kB 00:00 2024-03-13T07:24:34.869 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:24:34.877 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:24:34.877 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:24:34.929 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:24:34.930 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:24:35.087 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:24:35.153 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2007.gb7c579c9. 1/1 2024-03-13T07:24:35.178 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2007.gb7c579c9. 1/1 2024-03-13T07:24:35.178 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T07:24:35.178 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-13T07:24:35.178 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:35.812 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-2:19.0.0-2007.gb7c579c9.el9.x86_64 36/36 2024-03-13T07:24:35.812 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/36 2024-03-13T07:24:35.812 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mds-2:19.0.0-2007.gb7c579c9.el9.x86_64 2/36 2024-03-13T07:24:35.812 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-2:19.0.0-2007.gb7c579c9.el9.x86_64 3/36 2024-03-13T07:24:35.812 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mon-2:19.0.0-2007.gb7c579c9.el9.x86_64 4/36 2024-03-13T07:24:35.812 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-osd-2:19.0.0-2007.gb7c579c9.el9.x86_64 5/36 2024-03-13T07:24:35.812 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2007.gb7c579c9.el9. 6/36 2024-03-13T07:24:35.812 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-03-13T07:24:35.812 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-03-13T07:24:35.812 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-03-13T07:24:35.813 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-03-13T07:24:35.813 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-03-13T07:24:35.813 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-03-13T07:24:35.813 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-03-13T07:24:35.813 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-03-13T07:24:35.813 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-03-13T07:24:35.813 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/36 2024-03-13T07:24:35.813 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/36 2024-03-13T07:24:35.813 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-03-13T07:24:35.813 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-03-13T07:24:35.813 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-03-13T07:24:35.813 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-13T07:24:35.813 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-03-13T07:24:35.813 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-03-13T07:24:35.814 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-03-13T07:24:35.814 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-03-13T07:24:35.814 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-03-13T07:24:35.814 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-03-13T07:24:35.814 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-03-13T07:24:35.815 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-03-13T07:24:35.815 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-13T07:24:35.815 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-03-13T07:24:35.815 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-03-13T07:24:35.815 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-03-13T07:24:35.815 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-03-13T07:24:35.815 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-03-13T07:24:36.166 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2007.gb7c579c9. 1/1 2024-03-13T07:24:36.167 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:36.167 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:24:36.167 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:36.167 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:36.167 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:24:36.436 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-mgr 2024-03-13T07:24:36.736 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-03-13T07:24:36.737 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:36.737 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:24:36.737 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:36.737 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:36.737 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:36.737 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:24:36.737 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:36.737 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:36.737 INFO:teuthology.orchestra.run.smithi195.stdout: lua-5.4.4-4.el9.x86_64 2024-03-13T07:24:36.737 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-13T07:24:36.737 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-13T07:24:36.737 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-13T07:24:36.737 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-13T07:24:36.738 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-13T07:24:36.738 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-13T07:24:36.738 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-13T07:24:36.738 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-13T07:24:36.738 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-13T07:24:36.738 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-13T07:24:36.738 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-13T07:24:36.738 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-13T07:24:36.738 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-13T07:24:36.738 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-13T07:24:36.738 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-13T07:24:36.738 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-13T07:24:36.738 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-13T07:24:36.739 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply-3.11-14.el9.noarch 2024-03-13T07:24:36.739 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-13T07:24:36.739 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-13T07:24:36.739 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-13T07:24:36.739 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-13T07:24:36.739 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-13T07:24:36.739 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-13T07:24:36.739 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-13T07:24:36.739 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-13T07:24:36.739 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-13T07:24:36.739 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-13T07:24:36.739 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-13T07:24:36.739 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:36.739 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:24:37.016 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:01 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:24:37.114 INFO:teuthology.orchestra.run.smithi132.stdout:Package ceph-mgr-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:24:37.129 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-base 2024-03-13T07:24:37.166 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:24:37.167 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-03-13T07:24:37.167 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:24:37.228 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-mgr-dashboard 2024-03-13T07:24:37.701 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:00 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:24:37.788 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:01 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:24:37.798 INFO:teuthology.orchestra.run.smithi195.stdout:Package ceph-base-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:24:37.849 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:24:37.850 INFO:teuthology.orchestra.run.smithi195.stdout:Nothing to do. 2024-03-13T07:24:37.850 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:24:37.931 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install cephadm 2024-03-13T07:24:37.938 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:24:37.939 INFO:teuthology.orchestra.run.smithi132.stdout:====================================================================================== 2024-03-13T07:24:37.939 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-03-13T07:24:37.939 INFO:teuthology.orchestra.run.smithi132.stdout:====================================================================================== 2024-03-13T07:24:37.939 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:24:37.939 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 1.8 M 2024-03-13T07:24:37.939 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-03-13T07:24:37.939 INFO:teuthology.orchestra.run.smithi132.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-13T07:24:37.940 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 34 k 2024-03-13T07:24:37.940 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 16 k 2024-03-13T07:24:37.940 INFO:teuthology.orchestra.run.smithi132.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-13T07:24:37.940 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-13T07:24:37.940 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-13T07:24:37.940 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-13T07:24:37.940 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-13T07:24:37.940 INFO:teuthology.orchestra.run.smithi132.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-13T07:24:37.940 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-13T07:24:37.940 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-13T07:24:37.940 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:37.940 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:24:37.940 INFO:teuthology.orchestra.run.smithi132.stdout:====================================================================================== 2024-03-13T07:24:37.940 INFO:teuthology.orchestra.run.smithi132.stdout:Install 12 Packages 2024-03-13T07:24:37.940 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:37.941 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 6.9 M 2024-03-13T07:24:37.941 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 26 M 2024-03-13T07:24:37.941 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:24:38.456 INFO:teuthology.orchestra.run.smithi132.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2007.gb7 184 kB/s | 34 kB 00:00 2024-03-13T07:24:38.492 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:01 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:24:38.557 INFO:teuthology.orchestra.run.smithi132.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2007.gb7c 58 kB/s | 16 kB 00:00 2024-03-13T07:24:38.615 INFO:teuthology.orchestra.run.smithi132.stdout:(3/12): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-03-13T07:24:38.640 INFO:teuthology.orchestra.run.smithi132.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.8 MB/s | 238 kB 00:00 2024-03-13T07:24:38.641 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:24:38.641 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:24:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-03-13T07:24:38.641 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:24:38.641 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:24:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 748 k 2024-03-13T07:24:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:38.642 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:24:38.642 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:24:38.642 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:24:38.642 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:38.642 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 748 k 2024-03-13T07:24:38.642 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 755 k 2024-03-13T07:24:38.643 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:24:38.683 INFO:teuthology.orchestra.run.smithi132.stdout:(5/12): ceph-mgr-dashboard-19.0.0-2007.gb7c579c 4.4 MB/s | 1.8 MB 00:00 2024-03-13T07:24:38.791 INFO:teuthology.orchestra.run.smithi132.stdout:(6/12): protobuf-compiler-3.14.0-13.el9.x86_64. 4.8 MB/s | 863 kB 00:00 2024-03-13T07:24:38.927 INFO:teuthology.orchestra.run.smithi195.stdout:cephadm-19.0.0-2007.gb7c579c9.el9.noarch.rpm 2.6 MB/s | 748 kB 00:00 2024-03-13T07:24:38.928 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:24:38.928 INFO:teuthology.orchestra.run.smithi195.stdout:Total 2.6 MB/s | 748 kB 00:00 2024-03-13T07:24:38.928 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:24:38.934 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:24:38.934 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:24:38.939 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:24:38.940 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:24:38.975 INFO:teuthology.orchestra.run.smithi132.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 67 kB/s | 19 kB 00:00 2024-03-13T07:24:39.012 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:24:39.208 INFO:teuthology.orchestra.run.smithi132.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 966 kB/s | 548 kB 00:00 2024-03-13T07:24:39.275 INFO:teuthology.orchestra.run.smithi132.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 481 kB/s | 144 kB 00:00 2024-03-13T07:24:39.300 INFO:teuthology.orchestra.run.smithi132.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 334 kB/s | 31 kB 00:00 2024-03-13T07:24:39.418 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 1/1 2024-03-13T07:24:39.451 INFO:teuthology.orchestra.run.smithi132.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.1 MB/s | 188 kB 00:00 2024-03-13T07:24:39.506 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 1/1 2024-03-13T07:24:39.543 INFO:teuthology.orchestra.run.smithi132.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 2.7 MB/s | 2.0 MB 00:00 2024-03-13T07:24:39.545 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:24:39.546 INFO:teuthology.orchestra.run.smithi132.stdout:Total 4.3 MB/s | 6.9 MB 00:01 2024-03-13T07:24:39.563 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:24:39.580 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:24:39.580 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:24:39.759 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:24:39.759 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:24:39.917 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 1/1 2024-03-13T07:24:40.264 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 1/1 2024-03-13T07:24:40.264 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:40.264 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:24:40.264 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:24:40.264 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:40.264 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:24:40.270 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:24:40.351 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-03-13T07:24:40.387 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-03-13T07:24:40.433 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-03-13T07:24:40.463 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-immutable-object-cache 2024-03-13T07:24:40.464 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-03-13T07:24:40.571 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-03-13T07:24:40.696 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-03-13T07:24:40.783 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-03-13T07:24:40.835 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-03-13T07:24:40.864 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-03-13T07:24:40.896 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2007.gb7c579c9.el9 10/12 2024-03-13T07:24:41.023 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:04 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:24:41.137 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2007.gb7c579c9.el 11/12 2024-03-13T07:24:41.171 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2007.gb7c579c9.el9.noa 12/12 2024-03-13T07:24:41.177 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:24:41.178 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:24:41.178 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:24:41.178 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:24:41.178 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:24:41.178 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 145 k 2024-03-13T07:24:41.178 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:41.178 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:24:41.179 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:24:41.179 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:24:41.179 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:41.179 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 145 k 2024-03-13T07:24:41.179 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 436 k 2024-03-13T07:24:41.179 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:24:41.516 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-immutable-object-cache-19.0.0-2007.gb7c579 431 kB/s | 145 kB 00:00 2024-03-13T07:24:41.517 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:24:41.517 INFO:teuthology.orchestra.run.smithi195.stdout:Total 429 kB/s | 145 kB 00:00 2024-03-13T07:24:41.517 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:24:41.524 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:24:41.524 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:24:41.576 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:24:41.577 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:24:41.724 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:24:41.776 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2007.gb7c579c9. 1/1 2024-03-13T07:24:41.799 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2007.gb7c579c9. 1/1 2024-03-13T07:24:41.799 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T07:24:41.800 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-13T07:24:41.800 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:41.998 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2007.gb7c579c9.el9.noa 12/12 2024-03-13T07:24:41.998 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2007.gb7c579c9.el 1/12 2024-03-13T07:24:41.998 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2007.gb7c579c9.el9.noa 2/12 2024-03-13T07:24:41.998 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2007.gb7c579c9.el9 3/12 2024-03-13T07:24:41.998 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-03-13T07:24:41.999 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-03-13T07:24:41.999 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-03-13T07:24:41.999 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-03-13T07:24:41.999 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-03-13T07:24:41.999 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-03-13T07:24:41.999 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-03-13T07:24:41.999 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-03-13T07:24:42.365 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-03-13T07:24:42.365 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:42.365 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:24:42.365 INFO:teuthology.orchestra.run.smithi132.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-13T07:24:42.365 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:24:42.365 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:24:42.365 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:24:42.365 INFO:teuthology.orchestra.run.smithi132.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-13T07:24:42.365 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-13T07:24:42.365 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-13T07:24:42.365 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-13T07:24:42.365 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-13T07:24:42.365 INFO:teuthology.orchestra.run.smithi132.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-13T07:24:42.366 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-13T07:24:42.366 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-13T07:24:42.366 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:42.366 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:24:42.594 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-13T07:24:42.849 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2007.gb7c579c9. 1/1 2024-03-13T07:24:42.849 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:42.849 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:24:42.849 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:24:42.849 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:42.849 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:24:43.111 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-mgr 2024-03-13T07:24:43.160 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:07 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:24:43.311 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:24:43.312 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================================ 2024-03-13T07:24:43.312 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-03-13T07:24:43.312 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================================ 2024-03-13T07:24:43.312 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:24:43.312 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 7.4 M 2024-03-13T07:24:43.312 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-03-13T07:24:43.312 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-13T07:24:43.312 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-13T07:24:43.312 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-13T07:24:43.312 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-13T07:24:43.312 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-13T07:24:43.312 INFO:teuthology.orchestra.run.smithi132.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-13T07:24:43.312 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-13T07:24:43.313 INFO:teuthology.orchestra.run.smithi132.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-13T07:24:43.313 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-13T07:24:43.313 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-13T07:24:43.313 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-13T07:24:43.313 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:43.313 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:24:43.313 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================================ 2024-03-13T07:24:43.313 INFO:teuthology.orchestra.run.smithi132.stdout:Install 12 Packages 2024-03-13T07:24:43.313 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:43.314 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 37 M 2024-03-13T07:24:43.314 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 209 M 2024-03-13T07:24:43.314 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:24:43.684 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:06 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:24:43.781 INFO:teuthology.orchestra.run.smithi195.stdout:Package ceph-mgr-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:24:43.832 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:24:43.833 INFO:teuthology.orchestra.run.smithi195.stdout:Nothing to do. 2024-03-13T07:24:43.833 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:24:43.841 INFO:teuthology.orchestra.run.smithi132.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 182 kB/s | 32 kB 00:00 2024-03-13T07:24:43.900 INFO:teuthology.orchestra.run.smithi132.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 294 kB/s | 17 kB 00:00 2024-03-13T07:24:43.907 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-mgr-dashboard 2024-03-13T07:24:43.950 INFO:teuthology.orchestra.run.smithi132.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 686 kB/s | 34 kB 00:00 2024-03-13T07:24:44.093 INFO:teuthology.orchestra.run.smithi132.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.0 MB/s | 3.0 MB 00:00 2024-03-13T07:24:44.354 INFO:teuthology.orchestra.run.smithi132.stdout:(5/12): ceph-mgr-diskprediction-local-19.0.0-20 11 MB/s | 7.4 MB 00:00 2024-03-13T07:24:44.459 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:07 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:24:44.460 INFO:teuthology.orchestra.run.smithi132.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 9.3 MB/s | 4.7 MB 00:00 2024-03-13T07:24:44.535 INFO:teuthology.orchestra.run.smithi132.stdout:(7/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 4.8 MB 00:00 2024-03-13T07:24:44.560 INFO:teuthology.orchestra.run.smithi132.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.0 MB/s | 217 kB 00:00 2024-03-13T07:24:44.609 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:24:44.610 INFO:teuthology.orchestra.run.smithi195.stdout:====================================================================================== 2024-03-13T07:24:44.610 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-03-13T07:24:44.610 INFO:teuthology.orchestra.run.smithi195.stdout:====================================================================================== 2024-03-13T07:24:44.610 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:24:44.610 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 1.8 M 2024-03-13T07:24:44.610 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-03-13T07:24:44.610 INFO:teuthology.orchestra.run.smithi195.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-13T07:24:44.610 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 34 k 2024-03-13T07:24:44.611 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 16 k 2024-03-13T07:24:44.611 INFO:teuthology.orchestra.run.smithi195.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-13T07:24:44.611 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-13T07:24:44.611 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-13T07:24:44.611 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-13T07:24:44.611 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-13T07:24:44.611 INFO:teuthology.orchestra.run.smithi195.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-13T07:24:44.611 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-13T07:24:44.611 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-13T07:24:44.611 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:44.611 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:24:44.611 INFO:teuthology.orchestra.run.smithi195.stdout:====================================================================================== 2024-03-13T07:24:44.612 INFO:teuthology.orchestra.run.smithi195.stdout:Install 12 Packages 2024-03-13T07:24:44.612 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:44.612 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 6.9 M 2024-03-13T07:24:44.612 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 26 M 2024-03-13T07:24:44.612 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:24:45.094 INFO:teuthology.orchestra.run.smithi132.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-03-13T07:24:45.361 INFO:teuthology.orchestra.run.smithi132.stdout:(10/12): python3-devel-3.9.17-2.el9.x86_64.rpm 928 kB/s | 245 kB 00:00 2024-03-13T07:24:45.395 INFO:teuthology.orchestra.run.smithi195.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2007.gb7 180 kB/s | 34 kB 00:00 2024-03-13T07:24:45.503 INFO:teuthology.orchestra.run.smithi195.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2007.gb7c 55 kB/s | 16 kB 00:00 2024-03-13T07:24:45.620 INFO:teuthology.orchestra.run.smithi195.stdout:(3/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-03-13T07:24:45.646 INFO:teuthology.orchestra.run.smithi195.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.6 MB/s | 238 kB 00:00 2024-03-13T07:24:45.669 INFO:teuthology.orchestra.run.smithi132.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 176 kB/s | 195 kB 00:01 2024-03-13T07:24:45.704 INFO:teuthology.orchestra.run.smithi195.stdout:(5/12): ceph-mgr-dashboard-19.0.0-2007.gb7c579c 3.6 MB/s | 1.8 MB 00:00 2024-03-13T07:24:45.854 INFO:teuthology.orchestra.run.smithi195.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 154 kB/s | 19 kB 00:00 2024-03-13T07:24:45.930 INFO:teuthology.orchestra.run.smithi195.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.9 MB/s | 548 kB 00:00 2024-03-13T07:24:45.972 INFO:teuthology.orchestra.run.smithi195.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 3.4 MB/s | 144 kB 00:00 2024-03-13T07:24:46.014 INFO:teuthology.orchestra.run.smithi195.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 735 kB/s | 31 kB 00:00 2024-03-13T07:24:46.055 INFO:teuthology.orchestra.run.smithi195.stdout:(10/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 10 MB/s | 2.0 MB 00:00 2024-03-13T07:24:46.081 INFO:teuthology.orchestra.run.smithi195.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-03-13T07:24:46.478 INFO:teuthology.orchestra.run.smithi132.stdout:(12/12): libgfortran-11.4.1-3.el9.x86_64.rpm 419 kB/s | 803 kB 00:01 2024-03-13T07:24:46.481 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:24:46.481 INFO:teuthology.orchestra.run.smithi132.stdout:Total 12 MB/s | 37 MB 00:03 2024-03-13T07:24:46.523 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:24:46.549 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:24:46.549 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:24:46.905 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:24:46.906 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:24:47.289 INFO:teuthology.orchestra.run.smithi195.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 517 kB/s | 863 kB 00:01 2024-03-13T07:24:47.296 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:24:47.296 INFO:teuthology.orchestra.run.smithi195.stdout:Total 2.6 MB/s | 6.9 MB 00:02 2024-03-13T07:24:47.316 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:24:47.336 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:24:47.336 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:24:47.524 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:24:47.525 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:24:47.610 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:24:47.675 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-13T07:24:47.707 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-13T07:24:47.800 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-13T07:24:47.843 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-13T07:24:48.037 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:24:48.057 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-13T07:24:48.084 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-13T07:24:48.123 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-03-13T07:24:48.153 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-13T07:24:48.167 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-03-13T07:24:48.213 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-03-13T07:24:48.244 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-03-13T07:24:48.363 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-03-13T07:24:48.500 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-03-13T07:24:48.587 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-03-13T07:24:48.639 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-03-13T07:24:48.668 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-03-13T07:24:48.701 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2007.gb7c579c9.el9 10/12 2024-03-13T07:24:48.804 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-13T07:24:48.906 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-13T07:24:48.946 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2007.gb7c579c9.el 11/12 2024-03-13T07:24:48.975 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2007.gb7c579c9.el9.noa 12/12 2024-03-13T07:24:49.822 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2007.gb7c579c9.el9.noa 12/12 2024-03-13T07:24:49.822 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2007.gb7c579c9.el 1/12 2024-03-13T07:24:49.822 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2007.gb7c579c9.el9.noa 2/12 2024-03-13T07:24:49.823 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2007.gb7c579c9.el9 3/12 2024-03-13T07:24:49.823 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-03-13T07:24:49.823 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-03-13T07:24:49.823 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-03-13T07:24:49.823 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-03-13T07:24:49.823 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-03-13T07:24:49.823 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-03-13T07:24:49.823 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-03-13T07:24:49.823 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-03-13T07:24:50.145 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-13T07:24:50.209 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-03-13T07:24:50.210 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:50.210 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:24:50.210 INFO:teuthology.orchestra.run.smithi195.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-13T07:24:50.210 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:24:50.210 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:24:50.210 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:24:50.210 INFO:teuthology.orchestra.run.smithi195.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-13T07:24:50.210 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-13T07:24:50.210 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-13T07:24:50.210 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-13T07:24:50.210 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-13T07:24:50.211 INFO:teuthology.orchestra.run.smithi195.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-13T07:24:50.211 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-13T07:24:50.211 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-13T07:24:50.211 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:50.211 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:24:50.475 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-13T07:24:51.033 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:14 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:24:51.170 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-13T07:24:51.183 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:24:51.184 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================================ 2024-03-13T07:24:51.184 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-03-13T07:24:51.184 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================================ 2024-03-13T07:24:51.184 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:24:51.184 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 7.4 M 2024-03-13T07:24:51.184 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-03-13T07:24:51.185 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-13T07:24:51.185 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-13T07:24:51.185 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-13T07:24:51.185 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-13T07:24:51.185 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-13T07:24:51.185 INFO:teuthology.orchestra.run.smithi195.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-13T07:24:51.185 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-13T07:24:51.185 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-13T07:24:51.185 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-13T07:24:51.185 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-13T07:24:51.185 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-13T07:24:51.185 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:51.185 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:24:51.185 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================================ 2024-03-13T07:24:51.186 INFO:teuthology.orchestra.run.smithi195.stdout:Install 12 Packages 2024-03-13T07:24:51.186 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:24:51.186 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 37 M 2024-03-13T07:24:51.186 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 209 M 2024-03-13T07:24:51.186 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:24:51.210 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2007.gb7c57 12/12 2024-03-13T07:24:51.655 INFO:teuthology.orchestra.run.smithi195.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 214 kB/s | 32 kB 00:00 2024-03-13T07:24:51.688 INFO:teuthology.orchestra.run.smithi195.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 520 kB/s | 17 kB 00:00 2024-03-13T07:24:51.730 INFO:teuthology.orchestra.run.smithi195.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 823 kB/s | 34 kB 00:00 2024-03-13T07:24:51.889 INFO:teuthology.orchestra.run.smithi195.stdout:(4/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 30 MB/s | 4.7 MB 00:00 2024-03-13T07:24:52.014 INFO:teuthology.orchestra.run.smithi195.stdout:(5/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 39 MB/s | 4.8 MB 00:00 2024-03-13T07:24:52.073 INFO:teuthology.orchestra.run.smithi195.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.3 MB/s | 3.0 MB 00:00 2024-03-13T07:24:52.098 INFO:teuthology.orchestra.run.smithi195.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.6 MB/s | 217 kB 00:00 2024-03-13T07:24:52.207 INFO:teuthology.orchestra.run.smithi195.stdout:(8/12): ceph-mgr-diskprediction-local-19.0.0-20 11 MB/s | 7.4 MB 00:00 2024-03-13T07:24:52.465 INFO:teuthology.orchestra.run.smithi195.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-03-13T07:24:52.549 INFO:teuthology.orchestra.run.smithi195.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 575 kB/s | 195 kB 00:00 2024-03-13T07:24:52.608 INFO:teuthology.orchestra.run.smithi195.stdout:(11/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 803 kB 00:00 2024-03-13T07:24:52.641 INFO:teuthology.orchestra.run.smithi195.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-03-13T07:24:52.643 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:24:52.643 INFO:teuthology.orchestra.run.smithi195.stdout:Total 25 MB/s | 37 MB 00:01 2024-03-13T07:24:52.683 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:24:52.707 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:24:52.707 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:24:53.076 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:24:53.077 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:24:53.698 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2007.gb7c57 12/12 2024-03-13T07:24:53.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2007.gb7c57 1/12 2024-03-13T07:24:53.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-13T07:24:53.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-13T07:24:53.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-13T07:24:53.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-13T07:24:53.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-13T07:24:53.698 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-13T07:24:53.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-13T07:24:53.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-13T07:24:53.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-13T07:24:53.699 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-13T07:24:53.786 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:24:53.855 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-13T07:24:53.895 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-13T07:24:53.990 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-13T07:24:54.041 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-13T07:24:54.173 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-13T07:24:54.174 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:54.174 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:24:54.174 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:24:54.174 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-13T07:24:54.174 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-13T07:24:54.174 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-13T07:24:54.174 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-13T07:24:54.174 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-13T07:24:54.174 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-13T07:24:54.174 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-13T07:24:54.175 INFO:teuthology.orchestra.run.smithi132.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-13T07:24:54.175 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-13T07:24:54.175 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-13T07:24:54.175 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-13T07:24:54.175 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:54.175 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:24:54.216 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-13T07:24:54.247 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-13T07:24:54.325 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-13T07:24:54.529 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-mgr-rook 2024-03-13T07:24:54.985 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-13T07:24:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-13T07:24:55.103 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:19 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:24:55.253 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:24:55.254 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================== 2024-03-13T07:24:55.254 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-03-13T07:24:55.254 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================== 2024-03-13T07:24:55.254 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:24:55.255 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 49 k 2024-03-13T07:24:55.255 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-03-13T07:24:55.255 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-13T07:24:55.255 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-13T07:24:55.255 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-13T07:24:55.255 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-13T07:24:55.255 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-13T07:24:55.255 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-13T07:24:55.255 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-13T07:24:55.255 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-13T07:24:55.255 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-13T07:24:55.255 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-13T07:24:55.255 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-13T07:24:55.256 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-13T07:24:55.256 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:55.256 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:24:55.256 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================== 2024-03-13T07:24:55.256 INFO:teuthology.orchestra.run.smithi132.stdout:Install 13 Packages 2024-03-13T07:24:55.256 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:55.256 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 2.0 M 2024-03-13T07:24:55.257 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 26 M 2024-03-13T07:24:55.257 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:24:55.428 INFO:teuthology.orchestra.run.smithi132.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 202 kB/s | 24 kB 00:00 2024-03-13T07:24:55.453 INFO:teuthology.orchestra.run.smithi132.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 124 kB/s | 18 kB 00:00 2024-03-13T07:24:55.504 INFO:teuthology.orchestra.run.smithi132.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-03-13T07:24:55.528 INFO:teuthology.orchestra.run.smithi132.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-13T07:24:55.554 INFO:teuthology.orchestra.run.smithi132.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-03-13T07:24:55.579 INFO:teuthology.orchestra.run.smithi132.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 853 kB/s | 43 kB 00:00 2024-03-13T07:24:55.604 INFO:teuthology.orchestra.run.smithi132.stdout:(7/13): ceph-mgr-rook-19.0.0-2007.gb7c579c9.el9 166 kB/s | 49 kB 00:00 2024-03-13T07:24:55.946 INFO:teuthology.orchestra.run.smithi132.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 41 kB/s | 15 kB 00:00 2024-03-13T07:24:56.180 INFO:teuthology.orchestra.run.smithi132.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 51 kB/s | 32 kB 00:00 2024-03-13T07:24:56.355 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-13T07:24:56.434 INFO:teuthology.orchestra.run.smithi132.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 236 kB/s | 59 kB 00:00 2024-03-13T07:24:56.464 INFO:teuthology.orchestra.run.smithi132.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 2.0 MB/s | 1.0 MB 00:00 2024-03-13T07:24:56.489 INFO:teuthology.orchestra.run.smithi132.stdout:(12/13): python3-google-auth-2.28.0-1.el9.noarc 229 kB/s | 203 kB 00:00 2024-03-13T07:24:56.614 INFO:teuthology.orchestra.run.smithi132.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 488 kB/s | 90 kB 00:00 2024-03-13T07:24:56.618 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:24:56.619 INFO:teuthology.orchestra.run.smithi132.stdout:Total 1.5 MB/s | 2.0 MB 00:01 2024-03-13T07:24:56.619 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:24:56.632 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:24:56.632 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:24:56.733 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:24:56.733 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:24:57.265 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:24:57.372 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-13T07:24:57.397 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2007.gb7c57 12/12 2024-03-13T07:24:57.435 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-13T07:24:57.498 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-13T07:24:57.562 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-13T07:24:57.592 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-13T07:24:57.633 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-13T07:24:57.725 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-13T07:24:57.826 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 7/13 2024-03-13T07:24:57.885 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-13T07:24:58.600 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-13T07:24:58.672 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-13T07:24:58.708 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-13T07:24:58.755 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-13T07:24:58.777 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-rook-2:19.0.0-2007.gb7c579c9.el9.noarch 13/13 2024-03-13T07:24:59.594 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2007.gb7c579c9.el9.noarch 13/13 2024-03-13T07:24:59.595 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2007.gb7c579c9.el9.noarch 1/13 2024-03-13T07:24:59.595 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-13T07:24:59.595 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-13T07:24:59.595 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-13T07:24:59.595 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-13T07:24:59.595 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-13T07:24:59.595 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-13T07:24:59.595 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-13T07:24:59.595 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-13T07:24:59.595 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 10/13 2024-03-13T07:24:59.595 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-13T07:24:59.595 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-13T07:24:59.897 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2007.gb7c57 12/12 2024-03-13T07:24:59.897 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2007.gb7c57 1/12 2024-03-13T07:24:59.897 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-13T07:24:59.897 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-13T07:24:59.897 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-13T07:24:59.897 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-13T07:24:59.897 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-13T07:24:59.897 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-13T07:24:59.898 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-13T07:24:59.898 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-13T07:24:59.898 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-13T07:24:59.898 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-13T07:24:59.995 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-13T07:24:59.996 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:59.996 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:24:59.996 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:24:59.996 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-13T07:24:59.996 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-13T07:24:59.996 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-13T07:24:59.996 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-13T07:24:59.996 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-13T07:24:59.996 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-13T07:24:59.996 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-13T07:24:59.996 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-13T07:24:59.996 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-13T07:24:59.996 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-13T07:24:59.997 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-13T07:24:59.997 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-13T07:24:59.997 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:24:59.997 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:00.237 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-mgr-cephadm 2024-03-13T07:25:00.373 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-13T07:25:00.373 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:00.373 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:25:00.373 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:25:00.373 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-13T07:25:00.373 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-13T07:25:00.373 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-13T07:25:00.373 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-13T07:25:00.373 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-13T07:25:00.373 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-13T07:25:00.374 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-13T07:25:00.374 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-13T07:25:00.374 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-13T07:25:00.374 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-13T07:25:00.374 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-13T07:25:00.374 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:00.374 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:00.769 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-mgr-rook 2024-03-13T07:25:00.801 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:24 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:00.952 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:00.953 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================== 2024-03-13T07:25:00.953 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-03-13T07:25:00.953 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================== 2024-03-13T07:25:00.953 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:25:00.953 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 139 k 2024-03-13T07:25:00.953 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-03-13T07:25:00.954 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-13T07:25:00.954 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-13T07:25:00.954 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-13T07:25:00.954 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-13T07:25:00.954 INFO:teuthology.orchestra.run.smithi132.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-13T07:25:00.954 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:00.954 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:25:00.954 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================== 2024-03-13T07:25:00.954 INFO:teuthology.orchestra.run.smithi132.stdout:Install 6 Packages 2024-03-13T07:25:00.954 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:00.955 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 6.8 M 2024-03-13T07:25:00.955 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 33 M 2024-03-13T07:25:00.955 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:25:01.275 INFO:teuthology.orchestra.run.smithi132.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-03-13T07:25:01.334 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:24 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:01.367 INFO:teuthology.orchestra.run.smithi132.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2024-03-13T07:25:01.393 INFO:teuthology.orchestra.run.smithi132.stdout:(3/6): ceph-mgr-cephadm-19.0.0-2007.gb7c579c9.e 469 kB/s | 139 kB 00:00 2024-03-13T07:25:01.483 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:01.484 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================== 2024-03-13T07:25:01.485 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-03-13T07:25:01.485 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================== 2024-03-13T07:25:01.485 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:25:01.485 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 49 k 2024-03-13T07:25:01.485 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-03-13T07:25:01.485 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-13T07:25:01.485 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-13T07:25:01.485 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-13T07:25:01.485 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-13T07:25:01.485 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-13T07:25:01.485 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-13T07:25:01.485 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-13T07:25:01.486 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-13T07:25:01.486 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-13T07:25:01.486 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-13T07:25:01.486 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-13T07:25:01.486 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-13T07:25:01.486 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:01.486 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:25:01.486 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================== 2024-03-13T07:25:01.486 INFO:teuthology.orchestra.run.smithi195.stdout:Install 13 Packages 2024-03-13T07:25:01.486 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:01.487 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 2.0 M 2024-03-13T07:25:01.487 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 26 M 2024-03-13T07:25:01.487 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:25:01.689 INFO:teuthology.orchestra.run.smithi195.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 188 kB/s | 24 kB 00:00 2024-03-13T07:25:01.714 INFO:teuthology.orchestra.run.smithi195.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 117 kB/s | 18 kB 00:00 2024-03-13T07:25:01.717 INFO:teuthology.orchestra.run.smithi132.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 165 kB/s | 58 kB 00:00 2024-03-13T07:25:01.743 INFO:teuthology.orchestra.run.smithi132.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 146 kB/s | 51 kB 00:00 2024-03-13T07:25:01.764 INFO:teuthology.orchestra.run.smithi195.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-03-13T07:25:01.789 INFO:teuthology.orchestra.run.smithi195.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-13T07:25:01.815 INFO:teuthology.orchestra.run.smithi195.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-03-13T07:25:01.840 INFO:teuthology.orchestra.run.smithi195.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 855 kB/s | 43 kB 00:00 2024-03-13T07:25:01.865 INFO:teuthology.orchestra.run.smithi195.stdout:(7/13): ceph-mgr-rook-19.0.0-2007.gb7c579c9.el9 161 kB/s | 49 kB 00:00 2024-03-13T07:25:01.893 INFO:teuthology.orchestra.run.smithi132.stdout:(6/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 890 kB/s | 548 kB 00:00 2024-03-13T07:25:01.895 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:01.895 INFO:teuthology.orchestra.run.smithi132.stdout:Total 7.2 MB/s | 6.8 MB 00:00 2024-03-13T07:25:01.908 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:25:01.919 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:25:01.919 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:25:02.011 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:25:02.011 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:25:02.108 INFO:teuthology.orchestra.run.smithi195.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 56 kB/s | 15 kB 00:00 2024-03-13T07:25:02.133 INFO:teuthology.orchestra.run.smithi195.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 101 kB/s | 32 kB 00:00 2024-03-13T07:25:02.319 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:25:02.366 INFO:teuthology.orchestra.run.smithi195.stdout:(10/13): python3-google-auth-2.28.0-1.el9.noarc 404 kB/s | 203 kB 00:00 2024-03-13T07:25:02.475 INFO:teuthology.orchestra.run.smithi195.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 173 kB/s | 59 kB 00:00 2024-03-13T07:25:02.491 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-13T07:25:02.536 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-03-13T07:25:02.717 INFO:teuthology.orchestra.run.smithi195.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 256 kB/s | 90 kB 00:00 2024-03-13T07:25:02.867 INFO:teuthology.orchestra.run.smithi195.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.3 MB/s | 1.0 MB 00:00 2024-03-13T07:25:02.870 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:02.870 INFO:teuthology.orchestra.run.smithi195.stdout:Total 1.5 MB/s | 2.0 MB 00:01 2024-03-13T07:25:02.871 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:25:02.886 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:25:02.886 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:25:02.992 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:25:02.993 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:25:03.087 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-13T07:25:03.173 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-13T07:25:03.233 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-13T07:25:03.274 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 6/6 2024-03-13T07:25:03.514 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:25:03.671 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-13T07:25:03.726 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-13T07:25:03.781 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-13T07:25:03.810 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-13T07:25:03.852 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-13T07:25:03.944 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-13T07:25:04.046 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 7/13 2024-03-13T07:25:04.075 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 6/6 2024-03-13T07:25:04.075 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 1/6 2024-03-13T07:25:04.075 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-13T07:25:04.075 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-13T07:25:04.075 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-03-13T07:25:04.075 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-13T07:25:04.103 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-13T07:25:04.401 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-13T07:25:04.401 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:04.401 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:25:04.401 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:25:04.401 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-13T07:25:04.401 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-13T07:25:04.401 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-13T07:25:04.401 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-13T07:25:04.401 INFO:teuthology.orchestra.run.smithi132.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-13T07:25:04.401 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:04.401 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:04.635 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-fuse 2024-03-13T07:25:04.820 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-13T07:25:04.867 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-13T07:25:04.900 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-13T07:25:04.949 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-13T07:25:04.971 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-rook-2:19.0.0-2007.gb7c579c9.el9.noarch 13/13 2024-03-13T07:25:05.196 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:29 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:05.347 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:05.347 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:05.348 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-03-13T07:25:05.348 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:05.348 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:25:05.348 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 888 k 2024-03-13T07:25:05.348 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-03-13T07:25:05.348 INFO:teuthology.orchestra.run.smithi132.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-13T07:25:05.348 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:05.348 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:25:05.348 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:05.348 INFO:teuthology.orchestra.run.smithi132.stdout:Install 2 Packages 2024-03-13T07:25:05.348 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:05.349 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 967 k 2024-03-13T07:25:05.349 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 2.8 M 2024-03-13T07:25:05.349 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:25:05.480 INFO:teuthology.orchestra.run.smithi132.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 602 kB/s | 79 kB 00:00 2024-03-13T07:25:05.776 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2007.gb7c579c9.el9.noarch 13/13 2024-03-13T07:25:05.776 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2007.gb7c579c9.el9.noarch 1/13 2024-03-13T07:25:05.776 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-13T07:25:05.777 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-13T07:25:05.777 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-13T07:25:05.777 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-13T07:25:05.777 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-13T07:25:05.777 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-13T07:25:05.777 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-13T07:25:05.777 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-13T07:25:05.777 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 10/13 2024-03-13T07:25:05.778 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-13T07:25:05.778 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-13T07:25:05.838 INFO:teuthology.orchestra.run.smithi132.stdout:(2/2): ceph-fuse-19.0.0-2007.gb7c579c9.el9.x86_ 1.8 MB/s | 888 kB 00:00 2024-03-13T07:25:05.839 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:05.840 INFO:teuthology.orchestra.run.smithi132.stdout:Total 1.9 MB/s | 967 kB 00:00 2024-03-13T07:25:05.840 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:25:05.860 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:25:05.860 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:25:05.936 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:25:05.937 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:25:06.126 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:25:06.162 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-13T07:25:06.162 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:06.162 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:25:06.162 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:25:06.162 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-13T07:25:06.162 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-13T07:25:06.163 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-13T07:25:06.163 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-13T07:25:06.163 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-13T07:25:06.163 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-13T07:25:06.163 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-13T07:25:06.163 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-13T07:25:06.163 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-13T07:25:06.163 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-13T07:25:06.163 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-13T07:25:06.163 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-13T07:25:06.163 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:06.163 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:06.256 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-13T07:25:06.321 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-fuse-2:19.0.0-2007.gb7c579c9.el9.x86_64 2/2 2024-03-13T07:25:06.414 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-mgr-cephadm 2024-03-13T07:25:06.974 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:29 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:06.985 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2007.gb7c579c9.el9.x86_64 2/2 2024-03-13T07:25:06.985 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-fuse-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/2 2024-03-13T07:25:07.123 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:07.124 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================== 2024-03-13T07:25:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-03-13T07:25:07.124 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================== 2024-03-13T07:25:07.124 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:25:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 139 k 2024-03-13T07:25:07.124 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-03-13T07:25:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-13T07:25:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-13T07:25:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-13T07:25:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-13T07:25:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-13T07:25:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:07.124 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:25:07.125 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================== 2024-03-13T07:25:07.125 INFO:teuthology.orchestra.run.smithi195.stdout:Install 6 Packages 2024-03-13T07:25:07.125 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:07.125 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 6.8 M 2024-03-13T07:25:07.125 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 33 M 2024-03-13T07:25:07.126 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:25:07.351 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-13T07:25:07.351 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:07.351 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:25:07.351 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse-2:19.0.0-2007.gb7c579c9.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-13T07:25:07.351 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:07.351 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:07.635 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-volume 2024-03-13T07:25:07.769 INFO:teuthology.orchestra.run.smithi195.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-03-13T07:25:07.852 INFO:teuthology.orchestra.run.smithi195.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 5.8 MB 00:00 2024-03-13T07:25:07.952 INFO:teuthology.orchestra.run.smithi195.stdout:(3/6): ceph-mgr-cephadm-19.0.0-2007.gb7c579c9.e 339 kB/s | 139 kB 00:00 2024-03-13T07:25:08.206 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:32 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:08.219 INFO:teuthology.orchestra.run.smithi195.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 157 kB/s | 58 kB 00:00 2024-03-13T07:25:08.294 INFO:teuthology.orchestra.run.smithi195.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 149 kB/s | 51 kB 00:00 2024-03-13T07:25:08.358 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:08.358 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:08.358 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-03-13T07:25:08.358 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:08.358 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:25:08.358 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 264 k 2024-03-13T07:25:08.359 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-03-13T07:25:08.359 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-13T07:25:08.359 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:08.359 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:25:08.359 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:08.359 INFO:teuthology.orchestra.run.smithi132.stdout:Install 2 Packages 2024-03-13T07:25:08.359 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:08.360 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 332 k 2024-03-13T07:25:08.360 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 1.5 M 2024-03-13T07:25:08.360 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:25:08.461 INFO:teuthology.orchestra.run.smithi195.stdout:(6/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 792 kB/s | 548 kB 00:00 2024-03-13T07:25:08.463 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:08.464 INFO:teuthology.orchestra.run.smithi195.stdout:Total 5.1 MB/s | 6.8 MB 00:01 2024-03-13T07:25:08.478 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:25:08.489 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:25:08.490 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:25:08.493 INFO:teuthology.orchestra.run.smithi132.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 516 kB/s | 69 kB 00:00 2024-03-13T07:25:08.583 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:25:08.585 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:25:08.643 INFO:teuthology.orchestra.run.smithi132.stdout:(2/2): ceph-volume-19.0.0-2007.gb7c579c9.el9.no 930 kB/s | 264 kB 00:00 2024-03-13T07:25:08.644 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:08.645 INFO:teuthology.orchestra.run.smithi132.stdout:Total 1.1 MB/s | 332 kB 00:00 2024-03-13T07:25:08.645 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:25:08.654 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:25:08.654 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:25:08.696 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:25:08.696 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:25:08.845 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:25:08.876 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:25:09.026 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-03-13T07:25:09.040 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-13T07:25:09.062 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-volume-2:19.0.0-2007.gb7c579c9.el9.noarch 2/2 2024-03-13T07:25:09.078 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-volume-2:19.0.0-2007.gb7c579c9.el9.noarch 2/2 2024-03-13T07:25:09.078 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T07:25:09.078 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-13T07:25:09.078 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:09.087 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-03-13T07:25:09.595 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-13T07:25:09.681 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-13T07:25:09.751 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-13T07:25:09.775 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 6/6 2024-03-13T07:25:09.802 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-volume-2:19.0.0-2007.gb7c579c9.el9.noarch 1/2 2024-03-13T07:25:10.177 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-03-13T07:25:10.177 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:10.177 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:25:10.177 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:25:10.178 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-13T07:25:10.178 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:10.178 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:10.516 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install librados-devel 2024-03-13T07:25:10.551 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 6/6 2024-03-13T07:25:10.551 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 1/6 2024-03-13T07:25:10.551 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-13T07:25:10.552 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-13T07:25:10.552 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-03-13T07:25:10.552 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-13T07:25:10.876 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-13T07:25:10.876 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:10.876 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:25:10.876 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:25:10.876 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-13T07:25:10.876 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-13T07:25:10.876 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-13T07:25:10.876 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-13T07:25:10.876 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-13T07:25:10.876 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:10.876 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:11.092 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:35 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:11.125 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-fuse 2024-03-13T07:25:11.242 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:11.243 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:11.243 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:25:11.243 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:11.243 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:25:11.243 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 127 k 2024-03-13T07:25:11.243 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:11.243 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:25:11.243 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:11.243 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:25:11.244 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:11.244 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 127 k 2024-03-13T07:25:11.244 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 456 k 2024-03-13T07:25:11.244 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:25:11.474 INFO:teuthology.orchestra.run.smithi132.stdout:librados-devel-19.0.0-2007.gb7c579c9.el9.x86_64 550 kB/s | 127 kB 00:00 2024-03-13T07:25:11.475 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:11.475 INFO:teuthology.orchestra.run.smithi132.stdout:Total 547 kB/s | 127 kB 00:00 2024-03-13T07:25:11.475 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:25:11.481 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:25:11.481 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:25:11.536 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:25:11.536 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:25:11.676 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:34 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:11.681 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:25:11.791 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librados-devel-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:11.826 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:11.826 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:11.826 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-03-13T07:25:11.827 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:11.827 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:25:11.827 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 888 k 2024-03-13T07:25:11.827 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-03-13T07:25:11.827 INFO:teuthology.orchestra.run.smithi195.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-13T07:25:11.827 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:11.827 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:25:11.827 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:11.827 INFO:teuthology.orchestra.run.smithi195.stdout:Install 2 Packages 2024-03-13T07:25:11.827 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:11.828 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 967 k 2024-03-13T07:25:11.828 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 2.8 M 2024-03-13T07:25:11.828 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:25:11.946 INFO:teuthology.orchestra.run.smithi195.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 666 kB/s | 79 kB 00:00 2024-03-13T07:25:12.272 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados-devel-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:12.280 INFO:teuthology.orchestra.run.smithi195.stdout:(2/2): ceph-fuse-19.0.0-2007.gb7c579c9.el9.x86_ 1.9 MB/s | 888 kB 00:00 2024-03-13T07:25:12.281 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:12.281 INFO:teuthology.orchestra.run.smithi195.stdout:Total 2.1 MB/s | 967 kB 00:00 2024-03-13T07:25:12.281 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:25:12.301 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:25:12.301 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:25:12.377 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:25:12.377 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:25:12.557 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:25:12.606 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados-devel-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:12.607 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:12.607 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:25:12.607 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:25:12.607 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:12.607 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:12.698 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-13T07:25:12.770 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-fuse-2:19.0.0-2007.gb7c579c9.el9.x86_64 2/2 2024-03-13T07:25:12.860 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install libcephfs2 2024-03-13T07:25:13.427 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2007.gb7c579c9.el9.x86_64 2/2 2024-03-13T07:25:13.428 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-fuse-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/2 2024-03-13T07:25:13.429 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:37 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:13.529 INFO:teuthology.orchestra.run.smithi132.stdout:Package libcephfs2-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:25:13.582 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:13.583 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-03-13T07:25:13.583 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:13.650 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install libcephfs-devel 2024-03-13T07:25:13.769 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-13T07:25:13.769 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:13.769 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:25:13.769 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse-2:19.0.0-2007.gb7c579c9.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-13T07:25:13.769 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:13.769 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:14.063 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-volume 2024-03-13T07:25:14.216 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:38 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:14.367 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:14.367 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:14.367 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:25:14.367 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:14.368 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:25:14.368 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 31 k 2024-03-13T07:25:14.368 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:14.368 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:25:14.368 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:14.368 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:25:14.368 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:14.368 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 31 k 2024-03-13T07:25:14.368 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 140 k 2024-03-13T07:25:14.368 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:25:14.528 INFO:teuthology.orchestra.run.smithi132.stdout:libcephfs-devel-19.0.0-2007.gb7c579c9.el9.x86_6 195 kB/s | 31 kB 00:00 2024-03-13T07:25:14.528 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:14.529 INFO:teuthology.orchestra.run.smithi132.stdout:Total 194 kB/s | 31 kB 00:00 2024-03-13T07:25:14.529 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:25:14.532 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:25:14.532 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:25:14.561 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:25:14.562 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:25:14.633 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:37 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:14.680 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:25:14.768 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs-devel-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:14.785 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:14.785 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:14.785 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-03-13T07:25:14.785 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:14.786 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:25:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume noarch 2:19.0.0-2007.gb7c579c9.el9 ceph-noarch 264 k 2024-03-13T07:25:14.786 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-03-13T07:25:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-13T07:25:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:14.786 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:25:14.786 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:14.786 INFO:teuthology.orchestra.run.smithi195.stdout:Install 2 Packages 2024-03-13T07:25:14.786 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:14.787 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 332 k 2024-03-13T07:25:14.787 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 1.5 M 2024-03-13T07:25:14.787 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:25:14.926 INFO:teuthology.orchestra.run.smithi195.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 493 kB/s | 69 kB 00:00 2024-03-13T07:25:15.043 INFO:teuthology.orchestra.run.smithi195.stdout:(2/2): ceph-volume-19.0.0-2007.gb7c579c9.el9.no 1.0 MB/s | 264 kB 00:00 2024-03-13T07:25:15.044 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:15.044 INFO:teuthology.orchestra.run.smithi195.stdout:Total 1.3 MB/s | 332 kB 00:00 2024-03-13T07:25:15.045 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:25:15.053 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:25:15.053 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:25:15.094 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:25:15.094 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:25:15.136 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:15.244 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:25:15.424 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-03-13T07:25:15.434 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs-devel-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:15.434 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:15.435 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:25:15.435 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:25:15.435 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:15.435 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:15.454 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-volume-2:19.0.0-2007.gb7c579c9.el9.noarch 2/2 2024-03-13T07:25:15.470 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-volume-2:19.0.0-2007.gb7c579c9.el9.noarch 2/2 2024-03-13T07:25:15.470 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T07:25:15.470 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-13T07:25:15.470 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:15.629 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install librados2 2024-03-13T07:25:16.183 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-volume-2:19.0.0-2007.gb7c579c9.el9.noarch 1/2 2024-03-13T07:25:16.190 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:40 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:16.288 INFO:teuthology.orchestra.run.smithi132.stdout:Package librados2-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:25:16.341 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:16.342 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-03-13T07:25:16.342 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:16.407 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install librbd1 2024-03-13T07:25:16.525 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-03-13T07:25:16.525 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:16.526 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:25:16.526 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume-2:19.0.0-2007.gb7c579c9.el9.noarch 2024-03-13T07:25:16.526 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-13T07:25:16.526 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:16.526 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:16.841 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install librados-devel 2024-03-13T07:25:16.968 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:40 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:17.069 INFO:teuthology.orchestra.run.smithi132.stdout:Package librbd1-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:25:17.121 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:17.122 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-03-13T07:25:17.122 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:17.169 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install python3-rados 2024-03-13T07:25:17.417 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:40 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:17.567 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:17.568 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:17.568 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:25:17.568 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:17.569 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:25:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 127 k 2024-03-13T07:25:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:17.569 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:25:17.569 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:17.569 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:25:17.569 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:17.569 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 127 k 2024-03-13T07:25:17.569 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 456 k 2024-03-13T07:25:17.569 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:25:17.727 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:41 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:17.809 INFO:teuthology.orchestra.run.smithi195.stdout:librados-devel-19.0.0-2007.gb7c579c9.el9.x86_64 529 kB/s | 127 kB 00:00 2024-03-13T07:25:17.810 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:17.810 INFO:teuthology.orchestra.run.smithi195.stdout:Total 525 kB/s | 127 kB 00:00 2024-03-13T07:25:17.810 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:25:17.816 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:25:17.816 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:25:17.822 INFO:teuthology.orchestra.run.smithi132.stdout:Package python3-rados-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:25:17.871 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:25:17.872 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:25:17.873 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:17.875 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-03-13T07:25:17.875 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:17.928 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install python3-rgw 2024-03-13T07:25:18.012 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:25:18.099 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librados-devel-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:18.493 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:42 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:18.575 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados-devel-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:18.589 INFO:teuthology.orchestra.run.smithi132.stdout:Package python3-rgw-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:25:18.641 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:18.642 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-03-13T07:25:18.642 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:18.696 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install python3-cephfs 2024-03-13T07:25:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados-devel-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:18.934 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:25:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:25:18.934 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:18.934 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:19.195 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install libcephfs2 2024-03-13T07:25:19.263 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:43 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:19.359 INFO:teuthology.orchestra.run.smithi132.stdout:Package python3-cephfs-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:25:19.411 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:19.412 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-03-13T07:25:19.412 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:19.459 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install python3-rbd 2024-03-13T07:25:19.772 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:42 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:19.879 INFO:teuthology.orchestra.run.smithi195.stdout:Package libcephfs2-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:25:19.935 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:19.935 INFO:teuthology.orchestra.run.smithi195.stdout:Nothing to do. 2024-03-13T07:25:19.936 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:20.009 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install libcephfs-devel 2024-03-13T07:25:20.021 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:44 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:20.120 INFO:teuthology.orchestra.run.smithi132.stdout:Package python3-rbd-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:25:20.171 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:20.173 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-03-13T07:25:20.173 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:20.223 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install rbd-fuse 2024-03-13T07:25:20.737 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:43 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:20.787 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:44 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:20.938 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:20.938 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:20.939 INFO:teuthology.orchestra.run.smithi132.stdout: Package Architecture Version Repository Size 2024-03-13T07:25:20.939 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:20.939 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:25:20.939 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 85 k 2024-03-13T07:25:20.939 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:20.939 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:25:20.939 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:20.939 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:25:20.939 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:20.940 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 85 k 2024-03-13T07:25:20.940 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 231 k 2024-03-13T07:25:20.940 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:25:20.981 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:20.981 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:20.981 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:25:20.981 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:20.982 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:25:20.982 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 31 k 2024-03-13T07:25:20.982 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:20.982 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:25:20.982 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:20.982 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:25:20.982 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:20.983 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 31 k 2024-03-13T07:25:20.983 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 140 k 2024-03-13T07:25:20.983 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:25:21.146 INFO:teuthology.orchestra.run.smithi195.stdout:libcephfs-devel-19.0.0-2007.gb7c579c9.el9.x86_6 191 kB/s | 31 kB 00:00 2024-03-13T07:25:21.147 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:21.147 INFO:teuthology.orchestra.run.smithi195.stdout:Total 189 kB/s | 31 kB 00:00 2024-03-13T07:25:21.147 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:25:21.151 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:25:21.151 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:25:21.164 INFO:teuthology.orchestra.run.smithi132.stdout:rbd-fuse-19.0.0-2007.gb7c579c9.el9.x86_64.rpm 380 kB/s | 85 kB 00:00 2024-03-13T07:25:21.164 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:21.165 INFO:teuthology.orchestra.run.smithi132.stdout:Total 378 kB/s | 85 kB 00:00 2024-03-13T07:25:21.165 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:25:21.172 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:25:21.172 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:25:21.183 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:25:21.184 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:25:21.227 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:25:21.227 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:25:21.322 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:25:21.399 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:25:21.410 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs-devel-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:21.504 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-fuse-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:21.807 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:21.924 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:22.098 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs-devel-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:22.099 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:22.099 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:25:22.099 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:25:22.099 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:22.099 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:22.250 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-fuse-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:22.250 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:22.250 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:25:22.250 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:25:22.251 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:22.251 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:22.259 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install librados2 2024-03-13T07:25:22.495 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install rbd-mirror 2024-03-13T07:25:22.815 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:45 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:22.912 INFO:teuthology.orchestra.run.smithi195.stdout:Package librados2-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:25:22.964 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:22.965 INFO:teuthology.orchestra.run.smithi195.stdout:Nothing to do. 2024-03-13T07:25:22.965 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:23.017 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install librbd1 2024-03-13T07:25:23.091 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:47 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:23.246 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:23.247 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:23.247 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:25:23.247 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:23.247 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:25:23.247 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 3.1 M 2024-03-13T07:25:23.247 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:23.247 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:25:23.248 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:23.248 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:25:23.248 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:23.248 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 3.1 M 2024-03-13T07:25:23.248 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 13 M 2024-03-13T07:25:23.248 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:25:23.572 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:46 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:23.670 INFO:teuthology.orchestra.run.smithi195.stdout:Package librbd1-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:25:23.721 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:23.722 INFO:teuthology.orchestra.run.smithi195.stdout:Nothing to do. 2024-03-13T07:25:23.722 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:23.780 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install python3-rados 2024-03-13T07:25:24.118 INFO:teuthology.orchestra.run.smithi132.stdout:rbd-mirror-19.0.0-2007.gb7c579c9.el9.x86_64.rpm 3.6 MB/s | 3.1 MB 00:00 2024-03-13T07:25:24.119 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:24.119 INFO:teuthology.orchestra.run.smithi132.stdout:Total 3.5 MB/s | 3.1 MB 00:00 2024-03-13T07:25:24.119 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:25:24.129 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:25:24.129 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:25:24.203 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:25:24.203 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:25:24.338 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:47 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:24.434 INFO:teuthology.orchestra.run.smithi195.stdout:Package python3-rados-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:25:24.486 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:24.487 INFO:teuthology.orchestra.run.smithi195.stdout:Nothing to do. 2024-03-13T07:25:24.487 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:24.538 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install python3-rgw 2024-03-13T07:25:24.643 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:25:24.702 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-mirror-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:24.727 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:24.727 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T07:25:24.727 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-13T07:25:24.727 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-13T07:25:24.727 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-13T07:25:24.727 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:25.093 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:48 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:25.189 INFO:teuthology.orchestra.run.smithi195.stdout:Package python3-rgw-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:25:25.241 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:25.242 INFO:teuthology.orchestra.run.smithi195.stdout:Nothing to do. 2024-03-13T07:25:25.242 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:25.285 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install python3-cephfs 2024-03-13T07:25:25.846 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:48 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:25.918 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-mirror-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:25.918 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:25.918 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:25:25.918 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:25:25.918 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:25.919 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:25.943 INFO:teuthology.orchestra.run.smithi195.stdout:Package python3-cephfs-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:25:25.995 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:25.996 INFO:teuthology.orchestra.run.smithi195.stdout:Nothing to do. 2024-03-13T07:25:25.996 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:26.049 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install python3-rbd 2024-03-13T07:25:26.174 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install rbd-nbd 2024-03-13T07:25:26.606 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:49 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:26.702 INFO:teuthology.orchestra.run.smithi195.stdout:Package python3-rbd-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:25:26.748 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:50 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:26.754 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:26.755 INFO:teuthology.orchestra.run.smithi195.stdout:Nothing to do. 2024-03-13T07:25:26.755 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:26.807 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install rbd-fuse 2024-03-13T07:25:26.900 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:26.901 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:26.901 INFO:teuthology.orchestra.run.smithi132.stdout: Package Architecture Version Repository Size 2024-03-13T07:25:26.901 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:26.901 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:25:26.901 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 170 k 2024-03-13T07:25:26.901 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:26.901 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:25:26.901 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:26.901 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:25:26.901 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:26.901 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 170 k 2024-03-13T07:25:26.902 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 490 k 2024-03-13T07:25:26.902 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:25:27.263 INFO:teuthology.orchestra.run.smithi132.stdout:rbd-nbd-19.0.0-2007.gb7c579c9.el9.x86_64.rpm 471 kB/s | 170 kB 00:00 2024-03-13T07:25:27.264 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:27.264 INFO:teuthology.orchestra.run.smithi132.stdout:Total 469 kB/s | 170 kB 00:00 2024-03-13T07:25:27.264 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:25:27.272 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:25:27.272 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:25:27.328 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:25:27.329 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:25:27.364 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:50 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:27.478 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:25:27.514 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:27.515 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:27.515 INFO:teuthology.orchestra.run.smithi195.stdout: Package Architecture Version Repository Size 2024-03-13T07:25:27.515 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:27.515 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:25:27.515 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 85 k 2024-03-13T07:25:27.515 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:27.515 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:25:27.515 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:27.516 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:25:27.516 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:27.516 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 85 k 2024-03-13T07:25:27.516 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 231 k 2024-03-13T07:25:27.516 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:25:27.595 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-nbd-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:27.783 INFO:teuthology.orchestra.run.smithi195.stdout:rbd-fuse-19.0.0-2007.gb7c579c9.el9.x86_64.rpm 320 kB/s | 85 kB 00:00 2024-03-13T07:25:27.783 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:27.783 INFO:teuthology.orchestra.run.smithi195.stdout:Total 318 kB/s | 85 kB 00:00 2024-03-13T07:25:27.783 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:25:27.791 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:25:27.791 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:25:27.846 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:25:27.846 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:25:27.986 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:25:28.044 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:28.099 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-fuse-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:28.351 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-nbd-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:28.351 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:28.351 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:25:28.351 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:25:28.351 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:28.351 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:28.527 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:28.565 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-base-debuginfo 2024-03-13T07:25:28.863 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-fuse-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:28.864 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:28.864 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:25:28.864 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:25:28.864 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:28.864 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:29.112 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install rbd-mirror 2024-03-13T07:25:29.146 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:53 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:29.300 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:29.300 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:29.300 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:25:29.301 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:29.301 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:25:29.301 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 68 M 2024-03-13T07:25:29.301 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-03-13T07:25:29.301 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 23 M 2024-03-13T07:25:29.301 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:29.301 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:25:29.301 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:29.301 INFO:teuthology.orchestra.run.smithi132.stdout:Install 2 Packages 2024-03-13T07:25:29.301 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:29.302 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 91 M 2024-03-13T07:25:29.302 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 501 M 2024-03-13T07:25:29.302 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:25:29.682 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:52 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:29.838 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:29.838 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:29.838 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:25:29.838 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:29.838 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:25:29.838 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 3.1 M 2024-03-13T07:25:29.838 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:29.838 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:25:29.838 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:29.838 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:25:29.839 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:29.839 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 3.1 M 2024-03-13T07:25:29.839 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 13 M 2024-03-13T07:25:29.839 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:25:31.750 INFO:teuthology.orchestra.run.smithi132.stdout:(1/2): ceph-debuginfo-19.0.0-2007.gb7c579c9.el9 9.4 MB/s | 23 MB 00:02 2024-03-13T07:25:32.502 INFO:teuthology.orchestra.run.smithi195.stdout:rbd-mirror-19.0.0-2007.gb7c579c9.el9.x86_64.rpm 1.2 MB/s | 3.1 MB 00:02 2024-03-13T07:25:32.503 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:32.503 INFO:teuthology.orchestra.run.smithi195.stdout:Total 1.2 MB/s | 3.1 MB 00:02 2024-03-13T07:25:32.504 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:25:32.515 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:25:32.515 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:25:32.590 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:25:32.591 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:25:33.010 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:25:33.078 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-mirror-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:33.102 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:33.102 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T07:25:33.102 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-13T07:25:33.102 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-13T07:25:33.102 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-13T07:25:33.102 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:33.817 INFO:teuthology.orchestra.run.smithi132.stdout:(2/2): ceph-base-debuginfo-19.0.0-2007.gb7c579c 15 MB/s | 68 MB 00:04 2024-03-13T07:25:33.819 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:33.819 INFO:teuthology.orchestra.run.smithi132.stdout:Total 20 MB/s | 91 MB 00:04 2024-03-13T07:25:33.819 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:25:33.824 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:25:33.824 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:25:34.299 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-mirror-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:34.299 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:34.299 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:25:34.299 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:25:34.299 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:34.300 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:34.416 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:25:34.416 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:25:34.595 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install rbd-nbd 2024-03-13T07:25:35.166 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:01:58 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:35.316 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:35.317 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:35.317 INFO:teuthology.orchestra.run.smithi195.stdout: Package Architecture Version Repository Size 2024-03-13T07:25:35.317 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:35.317 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:25:35.318 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 170 k 2024-03-13T07:25:35.318 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:35.318 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:25:35.318 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:35.318 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:25:35.318 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:35.318 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 170 k 2024-03-13T07:25:35.318 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 490 k 2024-03-13T07:25:35.318 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:25:35.672 INFO:teuthology.orchestra.run.smithi195.stdout:rbd-nbd-19.0.0-2007.gb7c579c9.el9.x86_64.rpm 482 kB/s | 170 kB 00:00 2024-03-13T07:25:35.673 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:35.673 INFO:teuthology.orchestra.run.smithi195.stdout:Total 480 kB/s | 170 kB 00:00 2024-03-13T07:25:35.673 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:25:35.680 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:25:35.680 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:25:35.736 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:25:35.736 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:25:35.883 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:25:35.986 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-nbd-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:36.410 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:36.764 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-nbd-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:25:36.764 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:36.764 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:25:36.764 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:25:36.764 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:36.764 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:37.015 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-base-debuginfo 2024-03-13T07:25:37.561 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:25:37.586 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:02:00 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:37.737 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:37.737 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:37.738 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:25:37.738 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:37.738 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:25:37.738 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 68 M 2024-03-13T07:25:37.738 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-03-13T07:25:37.738 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 23 M 2024-03-13T07:25:37.738 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:37.738 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:25:37.738 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:37.738 INFO:teuthology.orchestra.run.smithi195.stdout:Install 2 Packages 2024-03-13T07:25:37.738 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:37.739 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 91 M 2024-03-13T07:25:37.739 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 501 M 2024-03-13T07:25:37.739 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:25:39.017 INFO:teuthology.orchestra.run.smithi195.stdout:(1/2): ceph-debuginfo-19.0.0-2007.gb7c579c9.el9 18 MB/s | 23 MB 00:01 2024-03-13T07:25:43.041 INFO:teuthology.orchestra.run.smithi195.stdout:(2/2): ceph-base-debuginfo-19.0.0-2007.gb7c579c 13 MB/s | 68 MB 00:05 2024-03-13T07:25:43.043 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:25:43.043 INFO:teuthology.orchestra.run.smithi195.stdout:Total 17 MB/s | 91 MB 00:05 2024-03-13T07:25:43.044 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:25:43.048 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:25:43.048 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:25:43.641 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:25:43.641 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:25:44.681 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/2 2024-03-13T07:25:44.778 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-base-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 2/2 2024-03-13T07:25:46.779 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:25:48.268 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-base-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 2/2 2024-03-13T07:25:48.268 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-base-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/2 2024-03-13T07:25:48.560 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2/2 2024-03-13T07:25:48.560 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:48.561 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:25:48.561 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:25:48.561 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:25:48.561 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:48.561 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:25:48.775 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-common-debuginfo 2024-03-13T07:25:49.345 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:02:13 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:25:49.495 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:25:49.495 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:49.495 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:25:49.495 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:49.495 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:25:49.495 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 423 M 2024-03-13T07:25:49.495 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:49.495 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:25:49.495 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:25:49.496 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:25:49.496 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:25:49.496 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 423 M 2024-03-13T07:25:49.496 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 1.7 G 2024-03-13T07:25:49.496 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:25:53.873 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/2 2024-03-13T07:25:53.950 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-base-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 2/2 2024-03-13T07:25:57.354 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-base-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 2/2 2024-03-13T07:25:57.354 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-base-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/2 2024-03-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2/2 2024-03-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:25:57.644 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:25:57.645 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:57.645 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:25:57.884 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-common-debuginfo 2024-03-13T07:25:58.457 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:02:21 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:25:58.616 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:25:58.617 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:58.617 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:25:58.617 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:58.617 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:25:58.617 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 423 M 2024-03-13T07:25:58.618 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:58.618 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:25:58.618 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:25:58.618 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:25:58.618 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:25:58.618 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 423 M 2024-03-13T07:25:58.618 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 1.7 G 2024-03-13T07:25:58.618 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:26:15.736 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-common-debuginfo-19.0.0-2007.gb7c579c9.el9 16 MB/s | 423 MB 00:26 2024-03-13T07:26:15.736 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:26:15.737 INFO:teuthology.orchestra.run.smithi132.stdout:Total 16 MB/s | 423 MB 00:26 2024-03-13T07:26:15.737 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:26:15.741 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:26:15.741 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:26:18.249 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:26:18.250 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:26:27.910 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-common-debuginfo-19.0.0-2007.gb7c579c9.el9 14 MB/s | 423 MB 00:29 2024-03-13T07:26:27.911 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:26:27.911 INFO:teuthology.orchestra.run.smithi195.stdout:Total 14 MB/s | 423 MB 00:29 2024-03-13T07:26:27.911 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:26:27.916 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:26:27.916 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:26:30.426 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:26:30.427 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:27:01.041 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:27:01.129 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-common-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:27:09.437 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:27:09.713 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-common-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:27:09.713 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:27:09.713 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:27:09.713 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:27:09.713 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:27:09.714 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:27:09.969 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2024-03-13T07:27:10.537 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:03:34 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:27:10.687 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:27:10.687 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================== 2024-03-13T07:27:10.687 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:27:10.687 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================== 2024-03-13T07:27:10.688 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:27:10.688 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 1.4 M 2024-03-13T07:27:10.688 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:27:10.688 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:27:10.688 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================== 2024-03-13T07:27:10.688 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:27:10.688 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:27:10.688 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 1.4 M 2024-03-13T07:27:10.688 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 6.5 M 2024-03-13T07:27:10.688 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:27:11.493 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-immutable-object-cache-debuginfo-19.0.0-20 1.8 MB/s | 1.4 MB 00:00 2024-03-13T07:27:11.493 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:27:11.494 INFO:teuthology.orchestra.run.smithi132.stdout:Total 1.8 MB/s | 1.4 MB 00:00 2024-03-13T07:27:11.494 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:27:11.497 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:27:11.497 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:27:11.541 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:27:11.541 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:27:11.812 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:27:11.910 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.0.0-2007. 1/1 2024-03-13T07:27:12.347 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:19.0.0-2007. 1/1 2024-03-13T07:27:12.638 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.0.0-2007. 1/1 2024-03-13T07:27:12.638 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:27:12.639 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:27:12.639 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:27:12.639 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:27:12.639 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:27:12.800 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-radosgw-debuginfo 2024-03-13T07:27:13.170 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:27:13.262 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-common-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:27:13.362 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:03:37 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:27:13.509 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:27:13.510 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:27:13.510 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:27:13.510 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:27:13.510 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:27:13.510 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 178 M 2024-03-13T07:27:13.510 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:27:13.510 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:27:13.510 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:27:13.510 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:27:13.510 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:27:13.511 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 178 M 2024-03-13T07:27:13.511 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 761 M 2024-03-13T07:27:13.511 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:27:21.351 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:27:21.638 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-common-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:27:21.639 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:27:21.639 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:27:21.639 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:27:21.639 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:27:21.639 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:27:21.884 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2024-03-13T07:27:22.445 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:03:45 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:27:22.595 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:27:22.596 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================== 2024-03-13T07:27:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:27:22.596 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================== 2024-03-13T07:27:22.596 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:27:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 1.4 M 2024-03-13T07:27:22.596 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:27:22.597 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:27:22.597 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================== 2024-03-13T07:27:22.597 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:27:22.597 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:27:22.597 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 1.4 M 2024-03-13T07:27:22.597 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 6.5 M 2024-03-13T07:27:22.597 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:27:23.374 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-immutable-object-cache-debuginfo-19.0.0-20 1.8 MB/s | 1.4 MB 00:00 2024-03-13T07:27:23.375 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:27:23.375 INFO:teuthology.orchestra.run.smithi195.stdout:Total 1.8 MB/s | 1.4 MB 00:00 2024-03-13T07:27:23.375 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:27:23.379 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:27:23.379 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:27:23.420 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:27:23.420 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:27:23.677 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:27:23.773 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.0.0-2007. 1/1 2024-03-13T07:27:24.189 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:19.0.0-2007. 1/1 2024-03-13T07:27:24.511 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.0.0-2007. 1/1 2024-03-13T07:27:24.511 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:27:24.512 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:27:24.512 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:27:24.512 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:27:24.512 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:27:24.697 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-radosgw-debuginfo 2024-03-13T07:27:25.258 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:03:48 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:27:25.405 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:27:25.406 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:27:25.406 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:27:25.406 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:27:25.407 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:27:25.407 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 178 M 2024-03-13T07:27:25.407 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:27:25.407 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:27:25.407 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:27:25.407 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:27:25.407 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:27:25.407 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 178 M 2024-03-13T07:27:25.407 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 761 M 2024-03-13T07:27:25.407 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:27:26.819 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-radosgw-debuginfo-19.0.0-2007.gb7c579c9.el 13 MB/s | 178 MB 00:13 2024-03-13T07:27:26.820 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:27:26.821 INFO:teuthology.orchestra.run.smithi132.stdout:Total 13 MB/s | 178 MB 00:13 2024-03-13T07:27:26.821 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:27:26.825 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:27:26.825 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:27:27.895 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:27:27.896 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:27:34.218 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-radosgw-debuginfo-19.0.0-2007.gb7c579c9.el 20 MB/s | 178 MB 00:08 2024-03-13T07:27:34.219 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:27:34.220 INFO:teuthology.orchestra.run.smithi195.stdout:Total 20 MB/s | 178 MB 00:08 2024-03-13T07:27:34.220 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:27:34.224 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:27:34.224 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:27:35.295 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:27:35.296 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:27:46.102 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:27:46.208 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-radosgw-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x 1/1 2024-03-13T07:27:51.079 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x 1/1 2024-03-13T07:27:51.359 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-radosgw-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x 1/1 2024-03-13T07:27:51.359 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:27:51.359 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:27:51.359 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:27:51.359 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:27:51.359 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:27:51.565 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-test-debuginfo 2024-03-13T07:27:52.167 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:04:16 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:27:52.324 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:27:52.325 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:27:52.325 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:27:52.325 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:27:52.325 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:27:52.325 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 677 M 2024-03-13T07:27:52.326 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:27:52.326 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:27:52.326 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:27:52.326 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:27:52.326 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:27:52.326 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 677 M 2024-03-13T07:27:52.326 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 3.1 G 2024-03-13T07:27:52.326 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:27:53.780 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:27:53.882 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-radosgw-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x 1/1 2024-03-13T07:27:58.660 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x 1/1 2024-03-13T07:27:58.959 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-radosgw-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x 1/1 2024-03-13T07:27:58.959 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:27:58.959 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:27:58.959 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:27:58.960 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:27:58.960 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:27:59.167 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-test-debuginfo 2024-03-13T07:27:59.734 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:04:22 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:27:59.883 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:27:59.883 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:27:59.884 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:27:59.884 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:27:59.884 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:27:59.884 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 677 M 2024-03-13T07:27:59.884 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:27:59.884 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:27:59.884 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:27:59.884 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:27:59.884 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:27:59.885 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 677 M 2024-03-13T07:27:59.885 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 3.1 G 2024-03-13T07:27:59.885 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:28:37.999 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-test-debuginfo-19.0.0-2007.gb7c579c9.el9.x 15 MB/s | 677 MB 00:45 2024-03-13T07:28:38.000 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:28:38.000 INFO:teuthology.orchestra.run.smithi132.stdout:Total 15 MB/s | 677 MB 00:45 2024-03-13T07:28:38.000 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:28:38.005 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:28:38.005 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:28:41.993 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:28:41.993 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:28:59.154 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-test-debuginfo-19.0.0-2007.gb7c579c9.el9.x 11 MB/s | 677 MB 00:59 2024-03-13T07:28:59.155 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:28:59.155 INFO:teuthology.orchestra.run.smithi195.stdout:Total 11 MB/s | 677 MB 00:59 2024-03-13T07:28:59.155 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:28:59.161 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:28:59.161 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:29:03.148 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:29:03.148 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:29:53.277 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:29:53.908 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-test-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:30:03.717 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-test-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:30:04.012 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-test-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:30:04.012 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:30:04.012 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:30:04.012 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:30:04.012 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:30:04.012 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:30:04.287 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-base-debuginfo 2024-03-13T07:30:04.858 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:06:28 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:30:04.955 INFO:teuthology.orchestra.run.smithi132.stdout:Package ceph-base-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:30:05.007 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:30:05.008 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-03-13T07:30:05.009 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:30:05.067 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-mgr-debuginfo 2024-03-13T07:30:05.633 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:06:29 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:30:05.781 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:30:05.782 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:30:05.782 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:30:05.783 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:30:05.783 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:30:05.783 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 20 M 2024-03-13T07:30:05.783 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:30:05.783 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:30:05.783 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:30:05.783 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:30:05.783 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:30:05.783 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 20 M 2024-03-13T07:30:05.783 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 104 M 2024-03-13T07:30:05.784 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:30:06.902 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-mgr-debuginfo-19.0.0-2007.gb7c579c9.el9.x8 18 MB/s | 20 MB 00:01 2024-03-13T07:30:06.903 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:30:06.903 INFO:teuthology.orchestra.run.smithi132.stdout:Total 18 MB/s | 20 MB 00:01 2024-03-13T07:30:06.904 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:30:06.907 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:30:06.907 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:30:07.060 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:30:07.060 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:30:09.332 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:30:09.434 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:30:10.458 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:30:10.757 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:30:10.757 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:30:10.757 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:30:10.757 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:30:10.757 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:30:10.757 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:30:10.928 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-mds-debuginfo 2024-03-13T07:30:11.500 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:06:35 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:30:11.650 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:30:11.651 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:30:11.651 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:30:11.651 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:30:11.651 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:30:11.651 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 49 M 2024-03-13T07:30:11.651 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:30:11.651 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:30:11.651 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:30:11.652 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:30:11.652 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:30:11.652 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 49 M 2024-03-13T07:30:11.652 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 177 M 2024-03-13T07:30:11.652 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:30:14.088 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-mds-debuginfo-19.0.0-2007.gb7c579c9.el9.x8 20 MB/s | 49 MB 00:02 2024-03-13T07:30:14.089 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:30:14.090 INFO:teuthology.orchestra.run.smithi132.stdout:Total 20 MB/s | 49 MB 00:02 2024-03-13T07:30:14.090 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:30:14.094 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:30:14.094 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:30:14.275 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:30:14.412 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:30:14.413 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:30:14.964 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-test-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:30:19.241 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:30:19.328 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mds-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:30:20.797 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mds-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:30:21.085 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mds-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:30:21.085 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:30:21.085 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:30:21.085 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:30:21.085 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:30:21.085 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:30:21.287 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-mon-debuginfo 2024-03-13T07:30:21.859 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:06:45 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:30:22.008 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:30:22.009 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:30:22.009 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:30:22.009 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:30:22.009 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:30:22.009 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 99 M 2024-03-13T07:30:22.009 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:30:22.009 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:30:22.009 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:30:22.009 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:30:22.009 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:30:22.009 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 99 M 2024-03-13T07:30:22.010 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 450 M 2024-03-13T07:30:22.010 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:30:23.109 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-test-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:30:23.402 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-test-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:30:23.402 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:30:23.402 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:30:23.402 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:30:23.402 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:30:23.402 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:30:23.700 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-base-debuginfo 2024-03-13T07:30:24.260 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:06:47 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:30:24.358 INFO:teuthology.orchestra.run.smithi195.stdout:Package ceph-base-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 is already installed. 2024-03-13T07:30:24.409 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:30:24.410 INFO:teuthology.orchestra.run.smithi195.stdout:Nothing to do. 2024-03-13T07:30:24.410 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:30:24.484 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-mgr-debuginfo 2024-03-13T07:30:25.049 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:06:48 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:30:25.197 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:30:25.198 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:30:25.198 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:30:25.198 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:30:25.198 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:30:25.198 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 20 M 2024-03-13T07:30:25.198 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:30:25.198 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:30:25.198 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:30:25.198 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:30:25.198 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:30:25.199 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 20 M 2024-03-13T07:30:25.199 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 104 M 2024-03-13T07:30:25.199 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:30:27.524 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-mon-debuginfo-19.0.0-2007.gb7c579c9.el9.x8 18 MB/s | 99 MB 00:05 2024-03-13T07:30:27.524 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:30:27.525 INFO:teuthology.orchestra.run.smithi132.stdout:Total 18 MB/s | 99 MB 00:05 2024-03-13T07:30:27.525 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:30:27.529 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:30:27.529 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:30:28.083 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-mgr-debuginfo-19.0.0-2007.gb7c579c9.el9.x8 6.9 MB/s | 20 MB 00:02 2024-03-13T07:30:28.084 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:30:28.084 INFO:teuthology.orchestra.run.smithi195.stdout:Total 6.9 MB/s | 20 MB 00:02 2024-03-13T07:30:28.084 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:30:28.088 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:30:28.088 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:30:28.143 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:30:28.143 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:30:28.239 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:30:28.240 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:30:30.497 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:30:30.606 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:30:31.597 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:30:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:30:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:30:31.883 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:30:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:30:31.883 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:30:31.883 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:30:32.078 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-mds-debuginfo 2024-03-13T07:30:32.646 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:06:55 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:30:32.795 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:30:32.797 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:30:32.797 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:30:32.797 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:30:32.797 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:30:32.797 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 49 M 2024-03-13T07:30:32.797 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:30:32.797 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:30:32.797 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:30:32.797 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:30:32.797 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:30:32.797 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 49 M 2024-03-13T07:30:32.797 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 177 M 2024-03-13T07:30:32.797 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:30:35.327 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-mds-debuginfo-19.0.0-2007.gb7c579c9.el9.x8 19 MB/s | 49 MB 00:02 2024-03-13T07:30:35.327 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:30:35.328 INFO:teuthology.orchestra.run.smithi195.stdout:Total 19 MB/s | 49 MB 00:02 2024-03-13T07:30:35.328 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:30:35.332 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:30:35.332 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:30:35.655 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:30:35.655 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:30:38.680 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:30:39.063 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mon-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:30:40.507 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:30:40.608 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mds-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:30:41.972 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mds-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:30:42.126 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mon-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:30:42.265 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mds-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:30:42.265 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:30:42.265 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:30:42.265 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:30:42.265 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:30:42.265 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:30:42.433 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mon-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:30:42.433 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:30:42.433 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:30:42.434 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:30:42.434 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:30:42.434 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:30:42.505 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-mon-debuginfo 2024-03-13T07:30:42.620 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-osd-debuginfo 2024-03-13T07:30:43.066 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:07:06 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:30:43.189 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:07:07 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:30:43.213 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:30:43.213 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:30:43.213 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:30:43.213 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:30:43.213 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:30:43.213 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 99 M 2024-03-13T07:30:43.214 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:30:43.214 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:30:43.214 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:30:43.214 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:30:43.214 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:30:43.214 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 99 M 2024-03-13T07:30:43.214 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 450 M 2024-03-13T07:30:43.215 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:30:43.337 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:30:43.338 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:30:43.338 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:30:43.338 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:30:43.338 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:30:43.338 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 289 M 2024-03-13T07:30:43.338 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:30:43.338 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:30:43.338 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:30:43.338 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:30:43.338 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:30:43.339 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 289 M 2024-03-13T07:30:43.339 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 1.2 G 2024-03-13T07:30:43.339 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:30:48.220 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-mon-debuginfo-19.0.0-2007.gb7c579c9.el9.x8 20 MB/s | 99 MB 00:05 2024-03-13T07:30:48.221 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:30:48.221 INFO:teuthology.orchestra.run.smithi195.stdout:Total 20 MB/s | 99 MB 00:05 2024-03-13T07:30:48.222 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:30:48.226 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:30:48.226 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:30:48.844 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:30:48.844 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:30:59.074 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:30:59.176 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mon-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:31:02.147 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mon-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:31:02.436 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mon-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:31:02.437 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:31:02.437 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:31:02.437 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:31:02.437 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:31:02.437 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:31:02.548 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-osd-debuginfo-19.0.0-2007.gb7c579c9.el9.x8 15 MB/s | 289 MB 00:19 2024-03-13T07:31:02.549 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:31:02.549 INFO:teuthology.orchestra.run.smithi132.stdout:Total 15 MB/s | 289 MB 00:19 2024-03-13T07:31:02.550 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:31:02.554 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:31:02.554 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:31:02.649 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-osd-debuginfo 2024-03-13T07:31:03.212 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:07:26 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:31:03.360 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:31:03.360 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:31:03.360 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:31:03.360 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:31:03.360 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:31:03.360 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 289 M 2024-03-13T07:31:03.360 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:31:03.361 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:31:03.361 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:31:03.361 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:31:03.361 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:31:03.361 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 289 M 2024-03-13T07:31:03.361 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 1.2 G 2024-03-13T07:31:03.361 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:31:04.273 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:31:04.273 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:31:16.154 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-osd-debuginfo-19.0.0-2007.gb7c579c9.el9.x8 23 MB/s | 289 MB 00:12 2024-03-13T07:31:16.154 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:31:16.155 INFO:teuthology.orchestra.run.smithi195.stdout:Total 23 MB/s | 289 MB 00:12 2024-03-13T07:31:16.155 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:31:16.159 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:31:16.159 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:31:17.875 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:31:17.875 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:31:33.714 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:31:33.809 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-osd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:31:41.528 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-osd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:31:41.831 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-osd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:31:41.831 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:31:41.831 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:31:41.832 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:31:41.832 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:31:41.832 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:31:42.043 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-fuse-debuginfo 2024-03-13T07:31:42.614 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:08:06 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:31:42.764 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:31:42.765 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:31:42.765 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:31:42.765 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:31:42.765 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:31:42.765 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 6.0 M 2024-03-13T07:31:42.765 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:31:42.765 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:31:42.765 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:31:42.765 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:31:42.765 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:31:42.765 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 6.0 M 2024-03-13T07:31:42.765 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 28 M 2024-03-13T07:31:42.766 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:31:43.272 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-fuse-debuginfo-19.0.0-2007.gb7c579c9.el9.x 12 MB/s | 6.0 MB 00:00 2024-03-13T07:31:43.273 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:31:43.273 INFO:teuthology.orchestra.run.smithi132.stdout:Total 12 MB/s | 6.0 MB 00:00 2024-03-13T07:31:43.273 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:31:43.277 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:31:43.277 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:31:43.347 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:31:43.347 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:31:44.089 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:31:44.221 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-fuse-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:31:44.807 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-fuse-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:31:45.107 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-fuse-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:31:45.107 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:31:45.107 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:31:45.107 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:31:45.107 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:31:45.107 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:31:45.316 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install librados-devel-debuginfo 2024-03-13T07:31:45.881 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:08:09 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:31:46.032 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:31:46.032 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:31:46.033 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:31:46.033 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:31:46.033 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:31:46.033 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 109 k 2024-03-13T07:31:46.033 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:31:46.033 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:31:46.033 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:31:46.033 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:31:46.033 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:31:46.033 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 109 k 2024-03-13T07:31:46.033 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 440 k 2024-03-13T07:31:46.033 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:31:46.384 INFO:teuthology.orchestra.run.smithi132.stdout:librados-devel-debuginfo-19.0.0-2007.gb7c579c9. 310 kB/s | 109 kB 00:00 2024-03-13T07:31:46.384 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:31:46.385 INFO:teuthology.orchestra.run.smithi132.stdout:Total 309 kB/s | 109 kB 00:00 2024-03-13T07:31:46.385 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:31:46.388 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:31:46.388 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:31:46.421 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:31:46.422 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:31:46.541 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:31:46.657 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librados-devel-debuginfo-2:19.0.0-2007.gb7c579c9.el9 1/1 2024-03-13T07:31:47.055 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados-devel-debuginfo-2:19.0.0-2007.gb7c579c9.el9 1/1 2024-03-13T07:31:47.250 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:31:47.349 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-osd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:31:47.380 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados-devel-debuginfo-2:19.0.0-2007.gb7c579c9.el9 1/1 2024-03-13T07:31:47.380 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:31:47.380 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:31:47.380 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:31:47.380 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:31:47.380 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:31:47.568 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install libcephfs2-debuginfo 2024-03-13T07:31:48.137 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:08:12 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:31:48.288 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:31:48.290 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:31:48.290 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:31:48.290 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:31:48.290 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:31:48.290 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 5.4 M 2024-03-13T07:31:48.290 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:31:48.290 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:31:48.290 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:31:48.290 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:31:48.290 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:31:48.291 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 5.4 M 2024-03-13T07:31:48.291 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 25 M 2024-03-13T07:31:48.291 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:31:48.854 INFO:teuthology.orchestra.run.smithi132.stdout:libcephfs2-debuginfo-19.0.0-2007.gb7c579c9.el9. 9.5 MB/s | 5.4 MB 00:00 2024-03-13T07:31:48.854 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:31:48.855 INFO:teuthology.orchestra.run.smithi132.stdout:Total 9.5 MB/s | 5.4 MB 00:00 2024-03-13T07:31:48.855 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:31:48.858 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:31:48.858 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:31:48.924 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:31:48.925 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:31:49.588 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:31:49.694 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs2-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86 1/1 2024-03-13T07:31:50.262 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephfs2-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86 1/1 2024-03-13T07:31:50.546 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs2-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86 1/1 2024-03-13T07:31:50.546 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:31:50.546 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:31:50.546 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:31:50.547 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:31:50.547 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:31:50.762 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install librados2-debuginfo 2024-03-13T07:31:51.331 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:08:15 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:31:51.480 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:31:51.481 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:31:51.481 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:31:51.481 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:31:51.481 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:31:51.481 INFO:teuthology.orchestra.run.smithi132.stdout: librados2-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 45 M 2024-03-13T07:31:51.481 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:31:51.481 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:31:51.482 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:31:51.482 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:31:51.482 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:31:51.482 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 45 M 2024-03-13T07:31:51.482 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 255 M 2024-03-13T07:31:51.482 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:31:54.331 INFO:teuthology.orchestra.run.smithi132.stdout:librados2-debuginfo-19.0.0-2007.gb7c579c9.el9.x 16 MB/s | 45 MB 00:02 2024-03-13T07:31:54.332 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:31:54.332 INFO:teuthology.orchestra.run.smithi132.stdout:Total 16 MB/s | 45 MB 00:02 2024-03-13T07:31:54.333 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:31:54.337 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:31:54.337 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:31:54.644 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:31:54.644 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:31:54.943 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-osd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:31:55.262 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-osd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:31:55.262 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:31:55.263 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:31:55.263 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:31:55.263 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:31:55.263 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:31:55.496 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-fuse-debuginfo 2024-03-13T07:31:56.060 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:08:19 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:31:56.209 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:31:56.210 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:31:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:31:56.210 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:31:56.210 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:31:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 6.0 M 2024-03-13T07:31:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:31:56.210 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:31:56.210 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:31:56.210 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:31:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:31:56.211 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 6.0 M 2024-03-13T07:31:56.211 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 28 M 2024-03-13T07:31:56.211 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:31:56.720 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-fuse-debuginfo-19.0.0-2007.gb7c579c9.el9.x 12 MB/s | 6.0 MB 00:00 2024-03-13T07:31:56.720 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:31:56.721 INFO:teuthology.orchestra.run.smithi195.stdout:Total 12 MB/s | 6.0 MB 00:00 2024-03-13T07:31:56.721 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:31:56.724 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:31:56.724 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:31:56.794 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:31:56.794 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:31:57.529 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:31:57.644 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-fuse-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:31:58.261 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-fuse-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:31:58.571 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-fuse-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:31:58.572 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:31:58.572 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:31:58.572 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:31:58.572 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:31:58.572 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:31:58.794 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install librados-devel-debuginfo 2024-03-13T07:31:59.353 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:08:22 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:31:59.506 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:31:59.506 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:31:59.506 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:31:59.506 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:31:59.507 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:31:59.507 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 109 k 2024-03-13T07:31:59.507 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:31:59.507 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:31:59.507 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:31:59.507 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:31:59.507 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:31:59.507 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 109 k 2024-03-13T07:31:59.507 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 440 k 2024-03-13T07:31:59.507 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:31:59.731 INFO:teuthology.orchestra.run.smithi195.stdout:librados-devel-debuginfo-19.0.0-2007.gb7c579c9. 480 kB/s | 109 kB 00:00 2024-03-13T07:31:59.732 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:31:59.732 INFO:teuthology.orchestra.run.smithi195.stdout:Total 477 kB/s | 109 kB 00:00 2024-03-13T07:31:59.732 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:31:59.735 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:31:59.735 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:31:59.749 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:31:59.769 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:31:59.769 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:31:59.837 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librados2-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:31:59.882 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:31:59.988 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librados-devel-debuginfo-2:19.0.0-2007.gb7c579c9.el9 1/1 2024-03-13T07:32:00.383 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados-devel-debuginfo-2:19.0.0-2007.gb7c579c9.el9 1/1 2024-03-13T07:32:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados-devel-debuginfo-2:19.0.0-2007.gb7c579c9.el9 1/1 2024-03-13T07:32:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:00.689 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:32:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:00.690 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:32:00.923 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install libcephfs2-debuginfo 2024-03-13T07:32:01.488 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:08:24 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:32:01.636 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:32:01.637 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:01.637 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:32:01.637 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:01.637 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:32:01.637 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 5.4 M 2024-03-13T07:32:01.637 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:01.637 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:32:01.637 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:01.637 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:32:01.638 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:01.638 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 5.4 M 2024-03-13T07:32:01.638 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 25 M 2024-03-13T07:32:01.638 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:32:01.747 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:32:02.051 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:32:02.051 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:02.051 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:32:02.051 INFO:teuthology.orchestra.run.smithi132.stdout: librados2-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:02.051 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:02.051 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:32:02.260 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install librbd1-debuginfo 2024-03-13T07:32:02.293 INFO:teuthology.orchestra.run.smithi195.stdout:libcephfs2-debuginfo-19.0.0-2007.gb7c579c9.el9. 8.2 MB/s | 5.4 MB 00:00 2024-03-13T07:32:02.293 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:02.294 INFO:teuthology.orchestra.run.smithi195.stdout:Total 8.2 MB/s | 5.4 MB 00:00 2024-03-13T07:32:02.294 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:32:02.297 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:32:02.297 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:32:02.362 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:32:02.363 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:32:02.831 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:08:26 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:32:02.980 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:32:02.981 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:32:02.981 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:02.981 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:32:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 52 M 2024-03-13T07:32:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:02.982 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:32:02.982 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:02.982 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:32:02.982 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:02.982 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 52 M 2024-03-13T07:32:02.982 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 308 M 2024-03-13T07:32:02.982 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:32:03.030 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:32:03.142 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs2-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86 1/1 2024-03-13T07:32:03.709 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephfs2-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86 1/1 2024-03-13T07:32:04.004 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs2-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86 1/1 2024-03-13T07:32:04.004 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:04.004 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:32:04.004 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:04.004 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:04.004 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:32:04.206 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install librados2-debuginfo 2024-03-13T07:32:04.775 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:08:27 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:32:04.924 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:32:04.924 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:04.924 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:32:04.925 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:04.925 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:32:04.925 INFO:teuthology.orchestra.run.smithi195.stdout: librados2-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 45 M 2024-03-13T07:32:04.925 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:04.925 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:32:04.925 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:04.925 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:32:04.925 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:04.925 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 45 M 2024-03-13T07:32:04.925 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 255 M 2024-03-13T07:32:04.925 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:32:12.525 INFO:teuthology.orchestra.run.smithi132.stdout:librbd1-debuginfo-19.0.0-2007.gb7c579c9.el9.x86 5.5 MB/s | 52 MB 00:09 2024-03-13T07:32:12.526 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:12.526 INFO:teuthology.orchestra.run.smithi132.stdout:Total 5.5 MB/s | 52 MB 00:09 2024-03-13T07:32:12.527 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:32:12.530 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:32:12.530 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:32:12.883 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:32:12.883 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:32:14.269 INFO:teuthology.orchestra.run.smithi195.stdout:librados2-debuginfo-19.0.0-2007.gb7c579c9.el9.x 4.8 MB/s | 45 MB 00:09 2024-03-13T07:32:14.270 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:14.271 INFO:teuthology.orchestra.run.smithi195.stdout:Total 4.8 MB/s | 45 MB 00:09 2024-03-13T07:32:14.271 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:32:14.275 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:32:14.275 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:32:14.586 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:32:14.586 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:32:18.916 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:32:19.181 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librbd1-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:32:19.727 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:32:19.827 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librados2-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:32:21.434 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:32:21.734 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:32:21.744 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:32:21.744 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:21.744 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:32:21.744 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:21.744 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:21.745 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:32:21.948 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install python3-cephfs-debuginfo 2024-03-13T07:32:22.028 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_ 1/1 2024-03-13T07:32:22.028 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:22.028 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:32:22.028 INFO:teuthology.orchestra.run.smithi195.stdout: librados2-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:22.028 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:22.029 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:32:22.252 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install librbd1-debuginfo 2024-03-13T07:32:22.509 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:08:46 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:32:22.658 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:32:22.659 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:22.659 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:32:22.659 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:22.659 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:32:22.659 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 615 k 2024-03-13T07:32:22.659 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:22.659 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:32:22.659 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:22.659 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:32:22.659 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:22.659 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 615 k 2024-03-13T07:32:22.660 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 2.1 M 2024-03-13T07:32:22.660 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:32:22.809 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:08:45 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:32:22.959 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:32:22.959 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:22.959 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:32:22.960 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:22.960 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:32:22.960 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 52 M 2024-03-13T07:32:22.960 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:22.960 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:32:22.960 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:22.960 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:32:22.960 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:22.960 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 52 M 2024-03-13T07:32:22.960 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 308 M 2024-03-13T07:32:22.960 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:32:23.086 INFO:teuthology.orchestra.run.smithi132.stdout:python3-cephfs-debuginfo-19.0.0-2007.gb7c579c9. 1.4 MB/s | 615 kB 00:00 2024-03-13T07:32:23.086 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:23.087 INFO:teuthology.orchestra.run.smithi132.stdout:Total 1.4 MB/s | 615 kB 00:00 2024-03-13T07:32:23.087 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:32:23.090 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:32:23.090 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:32:23.126 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:32:23.127 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:32:23.294 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:32:23.401 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cephfs-debuginfo-2:19.0.0-2007.gb7c579c9.el9 1/1 2024-03-13T07:32:23.810 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: python3-cephfs-debuginfo-2:19.0.0-2007.gb7c579c9.el9 1/1 2024-03-13T07:32:24.090 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cephfs-debuginfo-2:19.0.0-2007.gb7c579c9.el9 1/1 2024-03-13T07:32:24.091 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:24.091 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:32:24.091 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:24.091 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:24.091 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:32:24.266 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install python3-rados-debuginfo 2024-03-13T07:32:24.834 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:08:48 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:32:24.985 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:32:24.986 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:24.986 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:32:24.986 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:24.986 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:32:24.986 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 770 k 2024-03-13T07:32:24.986 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:24.986 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:32:24.986 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:24.987 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:32:24.987 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:24.987 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 770 k 2024-03-13T07:32:24.987 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 2.7 M 2024-03-13T07:32:24.987 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:32:25.872 INFO:teuthology.orchestra.run.smithi132.stdout:python3-rados-debuginfo-19.0.0-2007.gb7c579c9.e 869 kB/s | 770 kB 00:00 2024-03-13T07:32:25.873 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:25.873 INFO:teuthology.orchestra.run.smithi132.stdout:Total 868 kB/s | 770 kB 00:00 2024-03-13T07:32:25.873 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:32:25.878 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:32:25.878 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:32:25.915 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:32:25.915 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:32:26.022 INFO:teuthology.orchestra.run.smithi195.stdout:librbd1-debuginfo-19.0.0-2007.gb7c579c9.el9.x86 17 MB/s | 52 MB 00:03 2024-03-13T07:32:26.023 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:26.023 INFO:teuthology.orchestra.run.smithi195.stdout:Total 17 MB/s | 52 MB 00:03 2024-03-13T07:32:26.024 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:32:26.028 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:32:26.028 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:32:26.093 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:32:26.189 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rados-debuginfo-2:19.0.0-2007.gb7c579c9.el9. 1/1 2024-03-13T07:32:26.377 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:32:26.378 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:32:26.601 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: python3-rados-debuginfo-2:19.0.0-2007.gb7c579c9.el9. 1/1 2024-03-13T07:32:26.883 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rados-debuginfo-2:19.0.0-2007.gb7c579c9.el9. 1/1 2024-03-13T07:32:26.883 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:26.883 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:32:26.883 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:26.883 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:26.883 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:32:27.090 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install python3-rbd-debuginfo 2024-03-13T07:32:27.661 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:08:51 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:32:27.811 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:32:27.812 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:27.812 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:32:27.812 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:27.812 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:32:27.812 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 1.1 M 2024-03-13T07:32:27.812 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:27.812 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:32:27.813 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:27.813 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:32:27.813 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:27.813 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 1.1 M 2024-03-13T07:32:27.813 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 4.3 M 2024-03-13T07:32:27.813 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:32:28.150 INFO:teuthology.orchestra.run.smithi132.stdout:python3-rbd-debuginfo-19.0.0-2007.gb7c579c9.el9 3.3 MB/s | 1.1 MB 00:00 2024-03-13T07:32:28.151 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:28.151 INFO:teuthology.orchestra.run.smithi132.stdout:Total 3.3 MB/s | 1.1 MB 00:00 2024-03-13T07:32:28.151 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:32:28.155 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:32:28.155 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:32:28.195 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:32:28.196 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:32:28.416 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:32:28.516 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rbd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:32:28.931 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: python3-rbd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:32:29.217 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rbd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:32:29.217 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:29.217 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:32:29.217 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:29.217 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:29.217 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:32:29.418 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install python3-rgw-debuginfo 2024-03-13T07:32:29.982 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:08:53 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:32:30.132 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:32:30.133 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:30.133 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:32:30.133 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:30.133 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:32:30.133 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 337 k 2024-03-13T07:32:30.133 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:30.133 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:32:30.133 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:30.134 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:32:30.134 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:30.134 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 337 k 2024-03-13T07:32:30.134 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 1.0 M 2024-03-13T07:32:30.134 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:32:30.395 INFO:teuthology.orchestra.run.smithi132.stdout:python3-rgw-debuginfo-19.0.0-2007.gb7c579c9.el9 1.3 MB/s | 337 kB 00:00 2024-03-13T07:32:30.396 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:30.396 INFO:teuthology.orchestra.run.smithi132.stdout:Total 1.3 MB/s | 337 kB 00:00 2024-03-13T07:32:30.396 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:32:30.400 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:32:30.400 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:32:30.435 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:32:30.435 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:32:30.574 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:32:30.679 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rgw-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:32:31.064 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: python3-rgw-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:32:31.350 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rgw-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:32:31.350 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:31.350 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:32:31.350 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:31.351 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:31.351 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:32:31.568 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install rbd-fuse-debuginfo 2024-03-13T07:32:32.138 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:08:56 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:32:32.290 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:32:32.291 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:32.291 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:32:32.291 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:32.291 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:32:32.291 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 517 k 2024-03-13T07:32:32.291 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:32.291 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:32:32.291 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:32.291 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:32:32.291 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:32.291 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 517 k 2024-03-13T07:32:32.291 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 2.6 M 2024-03-13T07:32:32.291 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:32:32.386 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:32:32.479 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librbd1-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:32:32.567 INFO:teuthology.orchestra.run.smithi132.stdout:rbd-fuse-debuginfo-19.0.0-2007.gb7c579c9.el9.x8 1.8 MB/s | 517 kB 00:00 2024-03-13T07:32:32.567 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:32.568 INFO:teuthology.orchestra.run.smithi132.stdout:Total 1.8 MB/s | 517 kB 00:00 2024-03-13T07:32:32.568 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:32:32.571 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:32:32.571 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:32:32.608 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:32:32.608 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:32:32.767 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:32:32.867 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-fuse-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:32:33.254 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-fuse-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:32:33.541 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-fuse-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:32:33.541 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:33.541 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:32:33.541 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:33.541 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:33.541 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:32:33.750 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install rbd-mirror-debuginfo 2024-03-13T07:32:34.318 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:08:58 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:32:34.470 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:32:34.471 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:34.471 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:32:34.471 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:34.471 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:32:34.471 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 52 M 2024-03-13T07:32:34.471 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:34.471 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:32:34.471 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:34.471 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:32:34.471 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:34.471 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 52 M 2024-03-13T07:32:34.471 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 298 M 2024-03-13T07:32:34.471 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:32:34.699 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:32:35.016 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:32:35.016 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:35.016 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:32:35.017 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:35.017 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:35.017 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:32:35.230 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install python3-cephfs-debuginfo 2024-03-13T07:32:35.788 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:08:58 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:32:35.938 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:32:35.938 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:35.939 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:32:35.939 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:35.939 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:32:35.939 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 615 k 2024-03-13T07:32:35.939 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:35.939 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:32:35.939 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:35.939 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:32:35.939 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:35.939 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 615 k 2024-03-13T07:32:35.939 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 2.1 M 2024-03-13T07:32:35.939 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:32:36.733 INFO:teuthology.orchestra.run.smithi195.stdout:python3-cephfs-debuginfo-19.0.0-2007.gb7c579c9. 775 kB/s | 615 kB 00:00 2024-03-13T07:32:36.734 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:36.734 INFO:teuthology.orchestra.run.smithi195.stdout:Total 774 kB/s | 615 kB 00:00 2024-03-13T07:32:36.734 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:32:36.738 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:32:36.738 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:32:36.777 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:32:36.777 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:32:36.945 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:32:37.048 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cephfs-debuginfo-2:19.0.0-2007.gb7c579c9.el9 1/1 2024-03-13T07:32:37.448 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: python3-cephfs-debuginfo-2:19.0.0-2007.gb7c579c9.el9 1/1 2024-03-13T07:32:37.510 INFO:teuthology.orchestra.run.smithi132.stdout:rbd-mirror-debuginfo-19.0.0-2007.gb7c579c9.el9. 17 MB/s | 52 MB 00:03 2024-03-13T07:32:37.511 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:37.512 INFO:teuthology.orchestra.run.smithi132.stdout:Total 17 MB/s | 52 MB 00:03 2024-03-13T07:32:37.512 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:32:37.516 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:32:37.516 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:32:37.748 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cephfs-debuginfo-2:19.0.0-2007.gb7c579c9.el9 1/1 2024-03-13T07:32:37.749 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:37.749 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:32:37.749 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:37.749 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:37.749 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:32:37.853 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:32:37.854 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:32:37.946 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install python3-rados-debuginfo 2024-03-13T07:32:38.507 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:09:01 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:32:38.656 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:32:38.657 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:32:38.657 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:38.657 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:32:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 770 k 2024-03-13T07:32:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:38.657 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:32:38.657 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:38.657 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:32:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:38.658 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 770 k 2024-03-13T07:32:38.658 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 2.7 M 2024-03-13T07:32:38.658 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:32:38.962 INFO:teuthology.orchestra.run.smithi195.stdout:python3-rados-debuginfo-19.0.0-2007.gb7c579c9.e 2.5 MB/s | 770 kB 00:00 2024-03-13T07:32:38.962 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:38.962 INFO:teuthology.orchestra.run.smithi195.stdout:Total 2.5 MB/s | 770 kB 00:00 2024-03-13T07:32:38.962 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:32:38.965 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:32:38.966 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:32:39.004 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:32:39.004 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:32:39.185 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:32:39.277 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rados-debuginfo-2:19.0.0-2007.gb7c579c9.el9. 1/1 2024-03-13T07:32:39.668 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: python3-rados-debuginfo-2:19.0.0-2007.gb7c579c9.el9. 1/1 2024-03-13T07:32:39.972 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rados-debuginfo-2:19.0.0-2007.gb7c579c9.el9. 1/1 2024-03-13T07:32:39.972 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:39.972 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:32:39.972 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:39.972 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:39.972 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:32:40.184 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install python3-rbd-debuginfo 2024-03-13T07:32:40.752 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:09:03 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:32:40.904 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:32:40.905 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:40.905 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:32:40.905 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:40.905 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:32:40.905 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 1.1 M 2024-03-13T07:32:40.905 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:40.905 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:32:40.905 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:40.905 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:32:40.905 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:40.905 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 1.1 M 2024-03-13T07:32:40.905 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 4.3 M 2024-03-13T07:32:40.905 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:32:41.348 INFO:teuthology.orchestra.run.smithi195.stdout:python3-rbd-debuginfo-19.0.0-2007.gb7c579c9.el9 2.5 MB/s | 1.1 MB 00:00 2024-03-13T07:32:41.348 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:41.349 INFO:teuthology.orchestra.run.smithi195.stdout:Total 2.5 MB/s | 1.1 MB 00:00 2024-03-13T07:32:41.349 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:32:41.352 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:32:41.352 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:32:41.392 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:32:41.392 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:32:41.616 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:32:41.714 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rbd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:32:42.147 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: python3-rbd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:32:42.448 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rbd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:32:42.448 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:42.449 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:32:42.449 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:42.449 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:42.449 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:32:42.641 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install python3-rgw-debuginfo 2024-03-13T07:32:43.222 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:09:06 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:32:43.373 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:32:43.374 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:43.374 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:32:43.374 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:43.374 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:32:43.374 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 337 k 2024-03-13T07:32:43.374 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:43.374 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:32:43.375 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:43.375 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:32:43.375 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:43.375 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 337 k 2024-03-13T07:32:43.375 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 1.0 M 2024-03-13T07:32:43.375 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:32:43.694 INFO:teuthology.orchestra.run.smithi195.stdout:python3-rgw-debuginfo-19.0.0-2007.gb7c579c9.el9 1.0 MB/s | 337 kB 00:00 2024-03-13T07:32:43.694 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:43.694 INFO:teuthology.orchestra.run.smithi195.stdout:Total 1.0 MB/s | 337 kB 00:00 2024-03-13T07:32:43.694 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:32:43.697 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:32:43.697 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:32:43.732 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:32:43.732 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:32:43.784 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:32:43.867 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:32:43.958 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rgw-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:32:44.043 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-mirror-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86 1/1 2024-03-13T07:32:44.378 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: python3-rgw-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:32:44.682 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rgw-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x8 1/1 2024-03-13T07:32:44.683 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:44.683 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:32:44.683 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:44.683 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:44.683 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:32:44.903 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install rbd-fuse-debuginfo 2024-03-13T07:32:45.468 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:09:08 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:32:45.617 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:32:45.618 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:32:45.618 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:45.618 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:32:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 517 k 2024-03-13T07:32:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:45.618 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:32:45.618 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:45.618 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:32:45.618 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:45.619 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 517 k 2024-03-13T07:32:45.619 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 2.6 M 2024-03-13T07:32:45.619 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:32:46.013 INFO:teuthology.orchestra.run.smithi195.stdout:rbd-fuse-debuginfo-19.0.0-2007.gb7c579c9.el9.x8 1.3 MB/s | 517 kB 00:00 2024-03-13T07:32:46.013 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:46.013 INFO:teuthology.orchestra.run.smithi195.stdout:Total 1.3 MB/s | 517 kB 00:00 2024-03-13T07:32:46.014 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:32:46.017 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:32:46.017 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:32:46.053 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:32:46.053 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:32:46.157 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-mirror-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86 1/1 2024-03-13T07:32:46.213 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:32:46.312 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-fuse-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:32:46.466 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-mirror-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86 1/1 2024-03-13T07:32:46.466 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:46.466 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:32:46.466 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:46.466 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:46.466 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:32:46.667 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install rbd-nbd-debuginfo 2024-03-13T07:32:46.730 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: rbd-fuse-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:32:47.030 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-fuse-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_6 1/1 2024-03-13T07:32:47.030 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:47.030 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:32:47.030 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:47.030 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:47.030 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:32:47.232 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:09:11 ago on Wed 13 Mar 2024 07:23:36 AM UTC. 2024-03-13T07:32:47.234 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install rbd-mirror-debuginfo 2024-03-13T07:32:47.382 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-13T07:32:47.383 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:47.383 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-03-13T07:32:47.383 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:47.383 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-13T07:32:47.383 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 1.3 M 2024-03-13T07:32:47.384 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:47.384 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-13T07:32:47.384 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-13T07:32:47.384 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-03-13T07:32:47.384 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:47.384 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 1.3 M 2024-03-13T07:32:47.384 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 5.7 M 2024-03-13T07:32:47.384 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-13T07:32:47.802 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:09:10 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:32:47.955 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:32:47.956 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:47.956 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:32:47.956 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:47.956 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:32:47.956 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 52 M 2024-03-13T07:32:47.956 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:47.956 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:32:47.956 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:32:47.956 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:32:47.956 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:47.956 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 52 M 2024-03-13T07:32:47.956 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 298 M 2024-03-13T07:32:47.957 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:32:48.171 INFO:teuthology.orchestra.run.smithi132.stdout:rbd-nbd-debuginfo-19.0.0-2007.gb7c579c9.el9.x86 1.6 MB/s | 1.3 MB 00:00 2024-03-13T07:32:48.171 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:48.172 INFO:teuthology.orchestra.run.smithi132.stdout:Total 1.6 MB/s | 1.3 MB 00:00 2024-03-13T07:32:48.172 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-13T07:32:48.175 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-13T07:32:48.175 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-13T07:32:48.219 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-13T07:32:48.219 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-13T07:32:48.459 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-13T07:32:48.570 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-nbd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:32:49.007 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-nbd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:32:49.305 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-nbd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:32:49.305 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:49.305 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-13T07:32:49.305 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:49.305 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:32:49.305 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-13T07:32:49.521 DEBUG:teuthology.parallel:result is None 2024-03-13T07:32:50.449 INFO:teuthology.orchestra.run.smithi195.stdout:rbd-mirror-debuginfo-19.0.0-2007.gb7c579c9.el9. 21 MB/s | 52 MB 00:02 2024-03-13T07:32:50.450 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:32:50.450 INFO:teuthology.orchestra.run.smithi195.stdout:Total 21 MB/s | 52 MB 00:02 2024-03-13T07:32:50.450 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:32:50.455 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:32:50.455 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:32:50.796 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:32:50.796 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:32:56.730 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:32:56.831 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-mirror-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86 1/1 2024-03-13T07:32:58.906 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: rbd-mirror-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86 1/1 2024-03-13T07:32:59.204 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-mirror-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86 1/1 2024-03-13T07:32:59.205 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:59.205 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:32:59.205 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:32:59.205 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:32:59.205 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:32:59.413 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install rbd-nbd-debuginfo 2024-03-13T07:32:59.974 INFO:teuthology.orchestra.run.smithi195.stdout:Last metadata expiration check: 0:09:22 ago on Wed 13 Mar 2024 07:23:37 AM UTC. 2024-03-13T07:33:00.126 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-03-13T07:33:00.127 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:33:00.127 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repo Size 2024-03-13T07:33:00.128 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:33:00.128 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-03-13T07:33:00.128 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd-debuginfo x86_64 2:19.0.0-2007.gb7c579c9.el9 ceph 1.3 M 2024-03-13T07:33:00.128 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:00.128 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-03-13T07:33:00.128 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2024-03-13T07:33:00.128 INFO:teuthology.orchestra.run.smithi195.stdout:Install 1 Package 2024-03-13T07:33:00.128 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:00.128 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 1.3 M 2024-03-13T07:33:00.128 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 5.7 M 2024-03-13T07:33:00.128 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-03-13T07:33:00.601 INFO:teuthology.orchestra.run.smithi195.stdout:rbd-nbd-debuginfo-19.0.0-2007.gb7c579c9.el9.x86 2.7 MB/s | 1.3 MB 00:00 2024-03-13T07:33:00.602 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-03-13T07:33:00.602 INFO:teuthology.orchestra.run.smithi195.stdout:Total 2.7 MB/s | 1.3 MB 00:00 2024-03-13T07:33:00.602 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-03-13T07:33:00.605 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-03-13T07:33:00.605 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-03-13T07:33:00.647 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-03-13T07:33:00.648 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-03-13T07:33:00.891 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-03-13T07:33:01.000 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-nbd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:33:01.429 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: rbd-nbd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:33:01.744 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-nbd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 1/1 2024-03-13T07:33:01.745 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:01.745 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-03-13T07:33:01.745 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd-debuginfo-2:19.0.0-2007.gb7c579c9.el9.x86_64 2024-03-13T07:33:01.745 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:01.745 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-03-13T07:33:01.961 DEBUG:teuthology.parallel:result is None 2024-03-13T07:33:01.962 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b7c579c9c3ed531070a6aa96f74f80b53f80de32 2024-03-13T07:33:02.113 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-13T07:33:02.137 INFO:teuthology.orchestra.run.smithi132.stdout:19.0.0-2007.gb7c579c9.el9 2024-03-13T07:33:02.137 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2007.gb7c579c9.el9 2024-03-13T07:33:02.138 INFO:teuthology.task.install:The correct ceph version 19.0.0-2007.gb7c579c9 is installed. 2024-03-13T07:33:02.139 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b7c579c9c3ed531070a6aa96f74f80b53f80de32 2024-03-13T07:33:02.277 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-13T07:33:02.298 INFO:teuthology.orchestra.run.smithi195.stdout:19.0.0-2007.gb7c579c9.el9 2024-03-13T07:33:02.298 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2007.gb7c579c9.el9 2024-03-13T07:33:02.298 INFO:teuthology.task.install:The correct ceph version 19.0.0-2007.gb7c579c9 is installed. 2024-03-13T07:33:02.300 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-13T07:33:02.300 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:02.300 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-13T07:33:02.333 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:02.333 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-13T07:33:02.364 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-13T07:33:02.365 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:02.365 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/daemon-helper 2024-03-13T07:33:02.403 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-13T07:33:02.473 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:02.474 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/daemon-helper 2024-03-13T07:33:02.501 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-13T07:33:02.568 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-13T07:33:02.568 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:02.568 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-13T07:33:02.598 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-13T07:33:02.666 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:02.666 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-13T07:33:02.692 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-13T07:33:02.760 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-13T07:33:02.760 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:02.760 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/stdin-killer 2024-03-13T07:33:02.789 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-13T07:33:02.856 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:02.856 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/stdin-killer 2024-03-13T07:33:02.883 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-13T07:33:02.951 INFO:teuthology.run_tasks:Running task ceph... 2024-03-13T07:33:03.039 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-13T07:33:03.040 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 777 /var/log/ceph 2024-03-13T07:33:03.042 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 777 /var/log/ceph 2024-03-13T07:33:03.068 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-13T07:33:03.068 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-13T07:33:03.110 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-13T07:33:03.136 INFO:tasks.ceph:Creating extra log directories... 2024-03-13T07:33:03.136 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-13T07:33:03.178 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-13T07:33:03.256 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-13T07:33:03.256 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 120, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(SLOW_OPS\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(OBJECT_MISPLACED\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-13T07:33:03.257 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993', 'branch': 'wip-yuri8-testing-2024-03-11-1138', 'description': 'rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7594993', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri8-testing-2024-03-11-1138'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 120, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(SLOW_OPS\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(OBJECT_MISPLACED\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request'], 'sha1': 'b7c579c9c3ed531070a6aa96f74f80b53f80de32', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'b7c579c9c3ed531070a6aa96f74f80b53f80de32'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri8-testing-2024-03-11-1138', 'sha1': 'b7c579c9c3ed531070a6aa96f74f80b53f80de32'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 1266, 'sha1': 'b7c579c9c3ed531070a6aa96f74f80b53f80de32', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-yuri8-testing-2024-03-11-1138', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'b7c579c9c3ed531070a6aa96f74f80b53f80de32', 'targets': {'smithi132.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEFZB1Zl15oaEhsQoQkm4aIPLf9pprFtx9T0xmH3GkM7BJHh9G1TANIlJ/xMCqXsZ7bEm0hx6mFdZpYK3/Lq3jE=', 'smithi195.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3eaO3daDzXfRTyTOLae0YzBR4WELsoKJuVa+GGGYjFqxjdnHE6e5I67oohVXUtMNTEGwJME1e7kHpeYpVSJXs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'workunit': {'clients': {'client.0': ['rados/test.sh']}, 'env': {'ALLOW_TIMEOUTS': '1'}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-12_18:29:22', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-13T07:33:03.257 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-13T07:33:03.260 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-13T07:33:03.275 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m0777 -- /var/run/ceph 2024-03-13T07:33:03.315 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/run/ceph 2024-03-13T07:33:03.342 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:03.342 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/scratch_devs of=/dev/stdout 2024-03-13T07:33:03.396 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-13T07:33:03.396 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_1 2024-03-13T07:33:03.450 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-13T07:33:03.450 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T07:33:03.450 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-13T07:33:03.450 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T07:33:03.450 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T07:33:03.450 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-13 07:25:24.976010984 +0000 2024-03-13T07:33:03.451 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-13 07:22:54.061760627 +0000 2024-03-13T07:33:03.451 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-13 07:22:54.061760627 +0000 2024-03-13T07:33:03.451 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-13 07:22:54.061760627 +0000 2024-03-13T07:33:03.451 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-13T07:33:03.516 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-03-13T07:33:03.516 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-03-13T07:33:03.516 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000203792 s, 2.5 MB/s 2024-03-13T07:33:03.517 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-13T07:33:03.573 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_2 2024-03-13T07:33:03.627 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-13T07:33:03.627 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T07:33:03.627 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-03-13T07:33:03.627 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T07:33:03.627 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T07:33:03.627 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-13 07:25:24.977010958 +0000 2024-03-13T07:33:03.627 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-13 07:22:54.610746614 +0000 2024-03-13T07:33:03.627 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-13 07:22:54.610746614 +0000 2024-03-13T07:33:03.628 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-13 07:22:54.610746614 +0000 2024-03-13T07:33:03.628 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-13T07:33:03.692 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-03-13T07:33:03.693 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-03-13T07:33:03.693 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000242383 s, 2.1 MB/s 2024-03-13T07:33:03.694 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-13T07:33:03.750 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_3 2024-03-13T07:33:03.804 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-13T07:33:03.804 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T07:33:03.804 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-03-13T07:33:03.804 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T07:33:03.804 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T07:33:03.804 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-13 07:25:24.977010958 +0000 2024-03-13T07:33:03.804 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-13 07:22:54.970737426 +0000 2024-03-13T07:33:03.804 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-13 07:22:54.970737426 +0000 2024-03-13T07:33:03.804 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-13 07:22:54.970737426 +0000 2024-03-13T07:33:03.805 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-13T07:33:03.870 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-03-13T07:33:03.870 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-03-13T07:33:03.870 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000186673 s, 2.7 MB/s 2024-03-13T07:33:03.871 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-13T07:33:03.927 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_4 2024-03-13T07:33:03.982 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-13T07:33:03.982 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T07:33:03.982 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-13T07:33:03.982 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T07:33:03.982 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T07:33:03.982 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-13 07:25:24.978010933 +0000 2024-03-13T07:33:03.982 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-13 07:22:55.302728952 +0000 2024-03-13T07:33:03.982 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-13 07:22:55.302728952 +0000 2024-03-13T07:33:03.983 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-13 07:22:55.302728952 +0000 2024-03-13T07:33:03.983 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-13T07:33:04.048 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-03-13T07:33:04.048 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-03-13T07:33:04.048 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000207384 s, 2.5 MB/s 2024-03-13T07:33:04.050 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-13T07:33:04.105 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-13T07:33:04.105 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:04.105 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/scratch_devs of=/dev/stdout 2024-03-13T07:33:04.121 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-13T07:33:04.121 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_1 2024-03-13T07:33:04.176 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-13T07:33:04.176 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T07:33:04.176 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-13T07:33:04.176 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T07:33:04.176 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T07:33:04.176 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-03-13 07:25:33.342802894 +0000 2024-03-13T07:33:04.176 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-03-13 07:22:54.594041800 +0000 2024-03-13T07:33:04.176 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-03-13 07:22:54.594041800 +0000 2024-03-13T07:33:04.177 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-03-13 07:22:54.594041800 +0000 2024-03-13T07:33:04.177 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-13T07:33:04.242 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-03-13T07:33:04.242 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-03-13T07:33:04.242 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000102054 s, 5.0 MB/s 2024-03-13T07:33:04.243 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-13T07:33:04.299 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_2 2024-03-13T07:33:04.355 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-13T07:33:04.355 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T07:33:04.355 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-13T07:33:04.355 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T07:33:04.355 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T07:33:04.355 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-03-13 07:25:33.342802894 +0000 2024-03-13T07:33:04.355 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-03-13 07:22:54.932035136 +0000 2024-03-13T07:33:04.355 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-03-13 07:22:54.932035136 +0000 2024-03-13T07:33:04.355 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-03-13 07:22:54.932035136 +0000 2024-03-13T07:33:04.356 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-13T07:33:04.420 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-03-13T07:33:04.420 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-03-13T07:33:04.420 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 8.1122e-05 s, 6.3 MB/s 2024-03-13T07:33:04.421 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-13T07:33:04.476 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_3 2024-03-13T07:33:04.530 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-13T07:33:04.530 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T07:33:04.530 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-13T07:33:04.530 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T07:33:04.530 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T07:33:04.531 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-03-13 07:25:33.342802894 +0000 2024-03-13T07:33:04.531 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-03-13 07:22:55.276028354 +0000 2024-03-13T07:33:04.531 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-03-13 07:22:55.276028354 +0000 2024-03-13T07:33:04.531 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-03-13 07:22:55.276028354 +0000 2024-03-13T07:33:04.531 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-13T07:33:04.594 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-03-13T07:33:04.595 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-03-13T07:33:04.595 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 9.954e-05 s, 5.1 MB/s 2024-03-13T07:33:04.596 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-13T07:33:04.651 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_4 2024-03-13T07:33:04.705 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-13T07:33:04.706 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T07:33:04.706 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-13T07:33:04.706 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T07:33:04.706 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T07:33:04.706 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-03-13 07:25:33.343802874 +0000 2024-03-13T07:33:04.706 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-03-13 07:22:55.603021908 +0000 2024-03-13T07:33:04.706 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-03-13 07:22:55.603021908 +0000 2024-03-13T07:33:04.706 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-03-13 07:22:55.603021908 +0000 2024-03-13T07:33:04.706 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-13T07:33:04.771 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-03-13T07:33:04.771 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-03-13T07:33:04.771 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 7.8529e-05 s, 6.5 MB/s 2024-03-13T07:33:04.772 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-13T07:33:04.828 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-13T07:33:04.828 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi132.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-13T07:33:04.829 INFO:tasks.ceph:Generating config... 2024-03-13T07:33:04.830 INFO:tasks.ceph:[client] debug monc = 20 2024-03-13T07:33:04.830 INFO:tasks.ceph:[client] debug ms = 1 2024-03-13T07:33:04.830 INFO:tasks.ceph:[client] debug objecter = 20 2024-03-13T07:33:04.830 INFO:tasks.ceph:[client] debug rados = 20 2024-03-13T07:33:04.830 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-03-13T07:33:04.830 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-03-13T07:33:04.830 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-03-13T07:33:04.830 INFO:tasks.ceph:[global] ms type = async 2024-03-13T07:33:04.830 INFO:tasks.ceph:[global] osd heartbeat grace = 80 2024-03-13T07:33:04.831 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-13T07:33:04.831 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-13T07:33:04.831 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-13T07:33:04.831 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-13T07:33:04.831 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-13T07:33:04.831 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-03-13T07:33:04.831 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2024-03-13T07:33:04.831 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-03-13T07:33:04.831 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-03-13T07:33:04.831 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-03-13T07:33:04.831 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-03-13T07:33:04.831 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-03-13T07:33:04.831 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-03-13T07:33:04.831 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-13T07:33:04.832 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-13T07:33:04.832 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-13T07:33:04.832 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-13T07:33:04.832 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-03-13T07:33:04.832 INFO:tasks.ceph:[osd] debug bluefs = 1 2024-03-13T07:33:04.832 INFO:tasks.ceph:[osd] debug bluestore = 1 2024-03-13T07:33:04.832 INFO:tasks.ceph:[osd] debug monc = 20 2024-03-13T07:33:04.832 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-13T07:33:04.832 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-13T07:33:04.832 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-13T07:33:04.832 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-13T07:33:04.832 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-13T07:33:04.832 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-13T07:33:04.832 INFO:tasks.ceph:[osd] osd class default list = * 2024-03-13T07:33:04.832 INFO:tasks.ceph:[osd] osd class load list = * 2024-03-13T07:33:04.833 INFO:tasks.ceph:[osd] osd client watch timeout = 120 2024-03-13T07:33:04.833 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-03-13T07:33:04.833 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-03-13T07:33:04.833 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-13T07:33:04.833 INFO:tasks.ceph:[osd] osd fast shutdown = False 2024-03-13T07:33:04.833 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-03-13T07:33:04.833 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-03-13T07:33:04.833 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2024-03-13T07:33:04.833 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-13T07:33:04.833 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-03-13T07:33:04.833 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-03-13T07:33:04.833 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-03-13T07:33:04.833 INFO:tasks.ceph:Setting up mon.a... 2024-03-13T07:33:04.834 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-13T07:33:04.895 INFO:teuthology.orchestra.run.smithi132.stdout:creating /etc/ceph/ceph.keyring 2024-03-13T07:33:04.898 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-13T07:33:04.943 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-13T07:33:04.970 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.132'), ('mon.c', '[v2:172.21.15.132:3301,v1:172.21.15.132:6790]'), ('mon.b', '172.21.15.195')] 2024-03-13T07:33:04.970 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790],172.21.15.195', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 120, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-03-13T07:33:04.971 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:04.971 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-13T07:33:05.024 DEBUG:teuthology.orchestra.run.smithi132:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.132 --addv c '[v2:172.21.15.132:3301,v1:172.21.15.132:6790]' --add b 172.21.15.195 --print /home/ubuntu/cephtest/ceph.monmap 2024-03-13T07:33:05.109 INFO:teuthology.orchestra.run.smithi132.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-13T07:33:05.109 INFO:teuthology.orchestra.run.smithi132.stdout:monmaptool: generated fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:05.109 INFO:teuthology.orchestra.run.smithi132.stdout:setting min_mon_release = quincy 2024-03-13T07:33:05.110 INFO:teuthology.orchestra.run.smithi132.stdout:epoch 0 2024-03-13T07:33:05.110 INFO:teuthology.orchestra.run.smithi132.stdout:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:05.110 INFO:teuthology.orchestra.run.smithi132.stdout:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:05.110 INFO:teuthology.orchestra.run.smithi132.stdout:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:05.110 INFO:teuthology.orchestra.run.smithi132.stdout:min_mon_release 17 (quincy) 2024-03-13T07:33:05.110 INFO:teuthology.orchestra.run.smithi132.stdout:election_strategy: 1 2024-03-13T07:33:05.110 INFO:teuthology.orchestra.run.smithi132.stdout:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:05.110 INFO:teuthology.orchestra.run.smithi132.stdout:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:05.110 INFO:teuthology.orchestra.run.smithi132.stdout:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:05.110 INFO:teuthology.orchestra.run.smithi132.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-13T07:33:05.111 DEBUG:teuthology.orchestra.run.smithi132:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-13T07:33:05.126 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 74e4bc2f-6a5e-4423-a8c8-807970a2f96c... 2024-03-13T07:33:05.126 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-13T07:33:05.170 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-13T07:33:05.216 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2024-03-13T07:33:05.216 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2024-03-13T07:33:05.216 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-13T07:33:05.216 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2024-03-13T07:33:05.216 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.216 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2024-03-13T07:33:05.216 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.216 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2024-03-13T07:33:05.216 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.216 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2024-03-13T07:33:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2024-03-13T07:33:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2024-03-13T07:33:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2024-03-13T07:33:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-13T07:33:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2024-03-13T07:33:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2024-03-13T07:33:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-13T07:33:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2024-03-13T07:33:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2024-03-13T07:33:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2024-03-13T07:33:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2024-03-13T07:33:05.218 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2024-03-13T07:33:05.218 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2024-03-13T07:33:05.218 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2024-03-13T07:33:05.218 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2024-03-13T07:33:05.218 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-13T07:33:05.218 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-13T07:33:05.218 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2024-03-13T07:33:05.218 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.218 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-13T07:33:05.218 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.218 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2024-03-13T07:33:05.218 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.218 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2024-03-13T07:33:05.218 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.219 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2024-03-13T07:33:05.219 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2024-03-13T07:33:05.219 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2024-03-13T07:33:05.219 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790],172.21.15.195" 2024-03-13T07:33:05.219 INFO:teuthology.orchestra.run.smithi132.stdout: mon client directed command retry = 5 2024-03-13T07:33:05.219 INFO:teuthology.orchestra.run.smithi132.stdout: mon election default strategy = 1 2024-03-13T07:33:05.219 INFO:teuthology.orchestra.run.smithi132.stdout: ms inject socket failures = 5000 2024-03-13T07:33:05.219 INFO:teuthology.orchestra.run.smithi132.stdout: ms type = async 2024-03-13T07:33:05.219 INFO:teuthology.orchestra.run.smithi132.stdout: osd heartbeat grace = 80 2024-03-13T07:33:05.219 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:05.219 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.219 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2024-03-13T07:33:05.219 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2024-03-13T07:33:05.219 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.219 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2024-03-13T07:33:05.219 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2024-03-13T07:33:05.220 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2024-03-13T07:33:05.220 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.220 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2024-03-13T07:33:05.220 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2024-03-13T07:33:05.220 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.220 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2024-03-13T07:33:05.220 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2024-03-13T07:33:05.220 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2024-03-13T07:33:05.220 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug trim objects = true 2024-03-13T07:33:05.220 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.220 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2024-03-13T07:33:05.220 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2024-03-13T07:33:05.220 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.220 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2024-03-13T07:33:05.220 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.221 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2024-03-13T07:33:05.221 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.221 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2024-03-13T07:33:05.221 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.221 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2024-03-13T07:33:05.221 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2024-03-13T07:33:05.221 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.221 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2024-03-13T07:33:05.221 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2024-03-13T07:33:05.221 INFO:teuthology.orchestra.run.smithi132.stdout: bdev async discard = True 2024-03-13T07:33:05.221 INFO:teuthology.orchestra.run.smithi132.stdout: bdev enable discard = True 2024-03-13T07:33:05.221 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2024-03-13T07:33:05.221 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2024-03-13T07:33:05.221 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore zero block detection = True 2024-03-13T07:33:05.221 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 1 2024-03-13T07:33:05.222 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 1 2024-03-13T07:33:05.222 INFO:teuthology.orchestra.run.smithi132.stdout: debug monc = 20 2024-03-13T07:33:05.222 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-03-13T07:33:05.222 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2024-03-13T07:33:05.222 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 10 2024-03-13T07:33:05.222 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2024-03-13T07:33:05.222 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2024-03-13T07:33:05.222 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2024-03-13T07:33:05.222 INFO:teuthology.orchestra.run.smithi132.stdout: osd class default list = * 2024-03-13T07:33:05.223 INFO:teuthology.orchestra.run.smithi132.stdout: osd class load list = * 2024-03-13T07:33:05.223 INFO:teuthology.orchestra.run.smithi132.stdout: osd client watch timeout = 120 2024-03-13T07:33:05.223 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify cached snaps = True 2024-03-13T07:33:05.223 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify missing on start = True 2024-03-13T07:33:05.223 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2024-03-13T07:33:05.223 INFO:teuthology.orchestra.run.smithi132.stdout: osd fast shutdown = False 2024-03-13T07:33:05.223 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock override recovery settings = True 2024-03-13T07:33:05.223 INFO:teuthology.orchestra.run.smithi132.stdout: osd memory target = 1610612736 2024-03-13T07:33:05.224 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2024-03-13T07:33:05.224 INFO:teuthology.orchestra.run.smithi132.stdout: osd op queue = debug_random 2024-03-13T07:33:05.224 INFO:teuthology.orchestra.run.smithi132.stdout: osd op queue cut off = debug_random 2024-03-13T07:33:05.224 INFO:teuthology.orchestra.run.smithi132.stdout: osd shutdown pgref assert = True 2024-03-13T07:33:05.224 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.224 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2024-03-13T07:33:05.224 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-03-13T07:33:05.224 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2024-03-13T07:33:05.224 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-03-13T07:33:05.224 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-03-13T07:33:05.224 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-03-13T07:33:05.224 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-03-13T07:33:05.224 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2024-03-13T07:33:05.224 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.224 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2024-03-13T07:33:05.225 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-03-13T07:33:05.225 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-03-13T07:33:05.225 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2024-03-13T07:33:05.225 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-03-13T07:33:05.225 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2024-03-13T07:33:05.225 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2024-03-13T07:33:05.225 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-03-13T07:33:05.225 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2024-03-13T07:33:05.225 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2024-03-13T07:33:05.225 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-03-13T07:33:05.225 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.226 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-13T07:33:05.226 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2024-03-13T07:33:05.226 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2024-03-13T07:33:05.226 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.226 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2024-03-13T07:33:05.226 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-13T07:33:05.226 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-13T07:33:05.226 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.226 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2024-03-13T07:33:05.226 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2024-03-13T07:33:05.226 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.226 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2024-03-13T07:33:05.226 INFO:teuthology.orchestra.run.smithi132.stdout: mon min osdmap epochs = 50 2024-03-13T07:33:05.226 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd crush smoke test = False 2024-03-13T07:33:05.227 INFO:teuthology.orchestra.run.smithi132.stdout: mon osdmap full prune interval = 2 2024-03-13T07:33:05.227 INFO:teuthology.orchestra.run.smithi132.stdout: mon osdmap full prune min = 15 2024-03-13T07:33:05.227 INFO:teuthology.orchestra.run.smithi132.stdout: mon osdmap full prune txsize = 2 2024-03-13T07:33:05.227 INFO:teuthology.orchestra.run.smithi132.stdout: mon scrub interval = 300 2024-03-13T07:33:05.227 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on pool no app = False 2024-03-13T07:33:05.227 INFO:teuthology.orchestra.run.smithi132.stdout: paxos service trim min = 10 2024-03-13T07:33:05.227 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:05.227 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2024-03-13T07:33:05.227 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2024-03-13T07:33:05.227 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2024-03-13T07:33:05.227 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2024-03-13T07:33:05.227 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-13T07:33:05.227 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-13T07:33:05.227 INFO:teuthology.orchestra.run.smithi132.stdout: debug monc = 20 2024-03-13T07:33:05.227 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-03-13T07:33:05.227 INFO:teuthology.orchestra.run.smithi132.stdout: debug objecter = 20 2024-03-13T07:33:05.228 INFO:teuthology.orchestra.run.smithi132.stdout: debug rados = 20 2024-03-13T07:33:05.228 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2024-03-13T07:33:05.228 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2024-03-13T07:33:05.228 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2024-03-13T07:33:05.228 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2024-03-13T07:33:05.228 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2024-03-13T07:33:05.228 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-13T07:33:05.229 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2024-03-13T07:33:05.229 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.229 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2024-03-13T07:33:05.229 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.229 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2024-03-13T07:33:05.229 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.229 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2024-03-13T07:33:05.229 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2024-03-13T07:33:05.229 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.229 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2024-03-13T07:33:05.229 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2024-03-13T07:33:05.229 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.229 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-13T07:33:05.229 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2024-03-13T07:33:05.229 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.230 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2024-03-13T07:33:05.230 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-13T07:33:05.230 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.230 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2024-03-13T07:33:05.230 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.230 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2024-03-13T07:33:05.230 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2024-03-13T07:33:05.230 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2024-03-13T07:33:05.230 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2024-03-13T07:33:05.230 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2024-03-13T07:33:05.230 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2024-03-13T07:33:05.230 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2024-03-13T07:33:05.230 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-13T07:33:05.230 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-13T07:33:05.230 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2024-03-13T07:33:05.231 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.231 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-13T07:33:05.231 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.231 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2024-03-13T07:33:05.231 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.231 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2024-03-13T07:33:05.231 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.231 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2024-03-13T07:33:05.231 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2024-03-13T07:33:05.231 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2024-03-13T07:33:05.231 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790],172.21.15.195" 2024-03-13T07:33:05.231 INFO:teuthology.orchestra.run.smithi195.stdout: mon client directed command retry = 5 2024-03-13T07:33:05.231 INFO:teuthology.orchestra.run.smithi195.stdout: mon election default strategy = 1 2024-03-13T07:33:05.231 INFO:teuthology.orchestra.run.smithi195.stdout: ms inject socket failures = 5000 2024-03-13T07:33:05.231 INFO:teuthology.orchestra.run.smithi195.stdout: ms type = async 2024-03-13T07:33:05.232 INFO:teuthology.orchestra.run.smithi195.stdout: osd heartbeat grace = 80 2024-03-13T07:33:05.232 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:05.232 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.232 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2024-03-13T07:33:05.232 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2024-03-13T07:33:05.232 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.232 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2024-03-13T07:33:05.232 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2024-03-13T07:33:05.232 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2024-03-13T07:33:05.232 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.232 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2024-03-13T07:33:05.232 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2024-03-13T07:33:05.232 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.232 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2024-03-13T07:33:05.232 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2024-03-13T07:33:05.233 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2024-03-13T07:33:05.233 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug trim objects = true 2024-03-13T07:33:05.233 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.233 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2024-03-13T07:33:05.233 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2024-03-13T07:33:05.233 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.233 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2024-03-13T07:33:05.233 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.233 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2024-03-13T07:33:05.233 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.233 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2024-03-13T07:33:05.233 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.233 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2024-03-13T07:33:05.233 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2024-03-13T07:33:05.234 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.234 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2024-03-13T07:33:05.234 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2024-03-13T07:33:05.234 INFO:teuthology.orchestra.run.smithi195.stdout: bdev async discard = True 2024-03-13T07:33:05.234 INFO:teuthology.orchestra.run.smithi195.stdout: bdev enable discard = True 2024-03-13T07:33:05.234 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2024-03-13T07:33:05.234 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2024-03-13T07:33:05.234 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore zero block detection = True 2024-03-13T07:33:05.234 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 1 2024-03-13T07:33:05.234 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 1 2024-03-13T07:33:05.234 INFO:teuthology.orchestra.run.smithi195.stdout: debug monc = 20 2024-03-13T07:33:05.234 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-03-13T07:33:05.234 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2024-03-13T07:33:05.235 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 10 2024-03-13T07:33:05.235 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2024-03-13T07:33:05.235 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2024-03-13T07:33:05.235 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2024-03-13T07:33:05.235 INFO:teuthology.orchestra.run.smithi195.stdout: osd class default list = * 2024-03-13T07:33:05.235 INFO:teuthology.orchestra.run.smithi195.stdout: osd class load list = * 2024-03-13T07:33:05.235 INFO:teuthology.orchestra.run.smithi195.stdout: osd client watch timeout = 120 2024-03-13T07:33:05.235 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify cached snaps = True 2024-03-13T07:33:05.235 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify missing on start = True 2024-03-13T07:33:05.235 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2024-03-13T07:33:05.235 INFO:teuthology.orchestra.run.smithi195.stdout: osd fast shutdown = False 2024-03-13T07:33:05.235 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock override recovery settings = True 2024-03-13T07:33:05.235 INFO:teuthology.orchestra.run.smithi195.stdout: osd memory target = 1610612736 2024-03-13T07:33:05.235 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2024-03-13T07:33:05.235 INFO:teuthology.orchestra.run.smithi195.stdout: osd op queue = debug_random 2024-03-13T07:33:05.236 INFO:teuthology.orchestra.run.smithi195.stdout: osd op queue cut off = debug_random 2024-03-13T07:33:05.236 INFO:teuthology.orchestra.run.smithi195.stdout: osd shutdown pgref assert = True 2024-03-13T07:33:05.236 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.236 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2024-03-13T07:33:05.236 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-03-13T07:33:05.236 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2024-03-13T07:33:05.236 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-03-13T07:33:05.236 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-03-13T07:33:05.236 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-03-13T07:33:05.236 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-03-13T07:33:05.236 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2024-03-13T07:33:05.236 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.236 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2024-03-13T07:33:05.237 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-03-13T07:33:05.237 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-03-13T07:33:05.237 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2024-03-13T07:33:05.237 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-03-13T07:33:05.237 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2024-03-13T07:33:05.237 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2024-03-13T07:33:05.237 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-03-13T07:33:05.237 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2024-03-13T07:33:05.237 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2024-03-13T07:33:05.237 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-03-13T07:33:05.237 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.237 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-13T07:33:05.237 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2024-03-13T07:33:05.237 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2024-03-13T07:33:05.237 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.238 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2024-03-13T07:33:05.238 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-13T07:33:05.238 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-13T07:33:05.238 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.238 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2024-03-13T07:33:05.238 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2024-03-13T07:33:05.238 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.238 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2024-03-13T07:33:05.238 INFO:teuthology.orchestra.run.smithi195.stdout: mon min osdmap epochs = 50 2024-03-13T07:33:05.238 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd crush smoke test = False 2024-03-13T07:33:05.238 INFO:teuthology.orchestra.run.smithi195.stdout: mon osdmap full prune interval = 2 2024-03-13T07:33:05.238 INFO:teuthology.orchestra.run.smithi195.stdout: mon osdmap full prune min = 15 2024-03-13T07:33:05.238 INFO:teuthology.orchestra.run.smithi195.stdout: mon osdmap full prune txsize = 2 2024-03-13T07:33:05.238 INFO:teuthology.orchestra.run.smithi195.stdout: mon scrub interval = 300 2024-03-13T07:33:05.238 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on pool no app = False 2024-03-13T07:33:05.239 INFO:teuthology.orchestra.run.smithi195.stdout: paxos service trim min = 10 2024-03-13T07:33:05.239 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-13T07:33:05.239 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2024-03-13T07:33:05.239 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2024-03-13T07:33:05.239 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2024-03-13T07:33:05.239 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2024-03-13T07:33:05.239 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-13T07:33:05.239 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-13T07:33:05.239 INFO:teuthology.orchestra.run.smithi195.stdout: debug monc = 20 2024-03-13T07:33:05.239 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-03-13T07:33:05.239 INFO:teuthology.orchestra.run.smithi195.stdout: debug objecter = 20 2024-03-13T07:33:05.239 INFO:teuthology.orchestra.run.smithi195.stdout: debug rados = 20 2024-03-13T07:33:05.239 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2024-03-13T07:33:05.239 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2024-03-13T07:33:05.239 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2024-03-13T07:33:05.241 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-13T07:33:05.241 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-13T07:33:05.337 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-13T07:33:05.337 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:05.337 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-13T07:33:05.352 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:05.352 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-13T07:33:05.407 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi132.front.sepia.ceph.com 2024-03-13T07:33:05.407 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:05.407 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-13T07:33:05.407 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-13T07:33:05.487 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:05.487 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-13T07:33:05.541 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi195.front.sepia.ceph.com 2024-03-13T07:33:05.541 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:05.541 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-13T07:33:05.542 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-13T07:33:05.582 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:05.582 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-13T07:33:05.637 INFO:tasks.ceph:Setting up mon nodes... 2024-03-13T07:33:05.637 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-13T07:33:05.637 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-13T07:33:05.692 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-13T07:33:05.695 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-13T07:33:05.770 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-13T07:33:05.772 INFO:tasks.ceph:Setting up mds nodes... 2024-03-13T07:33:05.772 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-13T07:33:05.772 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-13T07:33:05.815 INFO:teuthology.orchestra.run.smithi132.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-13T07:33:05.830 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-13T07:33:05.872 INFO:teuthology.orchestra.run.smithi195.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-13T07:33:05.887 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-13T07:33:05.888 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi132.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-13T07:33:05.888 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-13T07:33:05.915 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-13T07:33:05.915 INFO:tasks.ceph:role: osd.0 2024-03-13T07:33:05.915 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi132.front.sepia.ceph.com 2024-03-13T07:33:05.915 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-13T07:33:05.989 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-13T07:33:05.989 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-13T07:33:05.989 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-13T07:33:05.989 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-13T07:33:05.989 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-13T07:33:05.989 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-03-13T07:33:05.989 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-13T07:33:05.989 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-13T07:33:05.989 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-13T07:33:05.989 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-13T07:33:06.505 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-03-13T07:33:06.507 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-03-13T07:33:06.507 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-13T07:33:06.643 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-13T07:33:06.675 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-13T07:33:06.742 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-13T07:33:06.743 INFO:tasks.ceph:role: osd.1 2024-03-13T07:33:06.743 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi132.front.sepia.ceph.com 2024-03-13T07:33:06.743 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-13T07:33:06.817 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-13T07:33:06.817 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-13T07:33:06.817 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-13T07:33:06.817 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-13T07:33:06.817 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-13T07:33:06.817 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-03-13T07:33:06.817 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-13T07:33:06.817 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-13T07:33:06.817 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-13T07:33:06.817 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-13T07:33:07.257 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-03-13T07:33:07.260 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-03-13T07:33:07.260 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-13T07:33:07.352 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-13T07:33:07.383 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-13T07:33:07.450 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-13T07:33:07.450 INFO:tasks.ceph:role: osd.2 2024-03-13T07:33:07.450 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi132.front.sepia.ceph.com 2024-03-13T07:33:07.450 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-13T07:33:07.525 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-13T07:33:07.525 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-13T07:33:07.526 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-13T07:33:07.526 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-13T07:33:07.526 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-13T07:33:07.526 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-03-13T07:33:07.526 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-13T07:33:07.526 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-13T07:33:07.526 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-13T07:33:07.526 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-13T07:33:08.022 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-03-13T07:33:08.024 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-03-13T07:33:08.024 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-13T07:33:08.141 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-13T07:33:08.171 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-13T07:33:08.238 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-13T07:33:08.238 INFO:tasks.ceph:role: osd.3 2024-03-13T07:33:08.238 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi132.front.sepia.ceph.com 2024-03-13T07:33:08.238 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-13T07:33:08.313 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-13T07:33:08.313 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-13T07:33:08.313 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-13T07:33:08.313 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-13T07:33:08.313 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-13T07:33:08.313 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-03-13T07:33:08.313 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-13T07:33:08.313 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-13T07:33:08.314 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-13T07:33:08.314 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-13T07:33:08.846 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-03-13T07:33:08.847 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-03-13T07:33:08.847 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-13T07:33:08.986 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-13T07:33:09.021 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-13T07:33:09.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:09.129+0000 7fb06401c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-13T07:33:09.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:09.129+0000 7fb06401c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-13T07:33:09.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:09.129+0000 7fb06401c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-13T07:33:09.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:09.129+0000 7fb06401c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-13T07:33:09.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:09.129+0000 7fb06401c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-13T07:33:09.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:09.130+0000 7fb06401c640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-13T07:33:11.979 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-13T07:33:12.010 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-13T07:33:12.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:12.101+0000 7f6aee714640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-13T07:33:12.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:12.101+0000 7f6aee714640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-13T07:33:12.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:12.101+0000 7f6aee714640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-13T07:33:12.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:12.101+0000 7f6aee714640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-13T07:33:12.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:12.101+0000 7f6aee714640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-13T07:33:12.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:12.102+0000 7f6aee714640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-13T07:33:14.942 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-13T07:33:14.973 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-13T07:33:15.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:15.065+0000 7f0965e0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-13T07:33:15.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:15.065+0000 7f0965e0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-13T07:33:15.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:15.065+0000 7f0965e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-13T07:33:15.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:15.065+0000 7f0965e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-13T07:33:15.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:15.065+0000 7f0965e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-13T07:33:15.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:15.066+0000 7f0965e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-13T07:33:17.906 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-13T07:33:17.937 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-13T07:33:18.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:18.028+0000 7f0e07c14640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-13T07:33:18.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:18.029+0000 7f0e07c14640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-13T07:33:18.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:18.029+0000 7f0e07c14640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-13T07:33:18.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:18.029+0000 7f0e07c14640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-13T07:33:18.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:18.029+0000 7f0e07c14640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-13T07:33:18.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:18.029+0000 7f0e07c14640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-13T07:33:20.880 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-13T07:33:20.911 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-13T07:33:20.943 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-13T07:33:20.943 INFO:tasks.ceph:role: osd.4 2024-03-13T07:33:20.943 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com 2024-03-13T07:33:20.943 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-13T07:33:21.014 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-13T07:33:21.014 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-13T07:33:21.014 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-13T07:33:21.014 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-13T07:33:21.014 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-13T07:33:21.014 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-03-13T07:33:21.014 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-13T07:33:21.014 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-13T07:33:21.014 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-13T07:33:21.014 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-13T07:33:21.046 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-03-13T07:33:21.048 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-03-13T07:33:21.049 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-13T07:33:21.128 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-13T07:33:21.202 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-13T07:33:21.271 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-13T07:33:21.272 INFO:tasks.ceph:role: osd.5 2024-03-13T07:33:21.272 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com 2024-03-13T07:33:21.272 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-13T07:33:21.339 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-13T07:33:21.339 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-13T07:33:21.339 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-13T07:33:21.339 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-13T07:33:21.339 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-13T07:33:21.339 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-03-13T07:33:21.339 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-13T07:33:21.339 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-13T07:33:21.339 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-13T07:33:21.340 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-13T07:33:21.372 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-03-13T07:33:21.374 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-03-13T07:33:21.374 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-13T07:33:21.446 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-13T07:33:21.517 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-13T07:33:21.585 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-13T07:33:21.585 INFO:tasks.ceph:role: osd.6 2024-03-13T07:33:21.585 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com 2024-03-13T07:33:21.585 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-13T07:33:21.651 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-13T07:33:21.651 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-13T07:33:21.652 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-13T07:33:21.652 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-13T07:33:21.652 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-13T07:33:21.652 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-03-13T07:33:21.652 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-13T07:33:21.652 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-13T07:33:21.652 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-13T07:33:21.652 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-13T07:33:21.683 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-03-13T07:33:21.685 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-03-13T07:33:21.685 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-13T07:33:21.757 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-13T07:33:21.829 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-13T07:33:21.897 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-13T07:33:21.897 INFO:tasks.ceph:role: osd.7 2024-03-13T07:33:21.897 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com 2024-03-13T07:33:21.897 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-13T07:33:21.963 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-13T07:33:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-13T07:33:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-13T07:33:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-13T07:33:21.964 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-13T07:33:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-03-13T07:33:21.964 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-13T07:33:21.964 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-13T07:33:21.964 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-13T07:33:21.964 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-13T07:33:21.995 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-03-13T07:33:21.997 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-03-13T07:33:21.997 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-13T07:33:22.067 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-13T07:33:22.139 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-13T07:33:22.234 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:22.234+0000 7fb16cfbd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-13T07:33:22.234 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:22.234+0000 7fb16cfbd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-13T07:33:22.235 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:22.234+0000 7fb16cfbd640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-13T07:33:22.235 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:22.234+0000 7fb16cfbd640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-13T07:33:22.235 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:22.234+0000 7fb16cfbd640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-13T07:33:22.235 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:22.234+0000 7fb16cfbd640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-13T07:33:25.073 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-13T07:33:25.104 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-13T07:33:25.195 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:25.194+0000 7f8a9c437640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-13T07:33:25.195 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:25.194+0000 7f8a9c437640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-13T07:33:25.195 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:25.194+0000 7f8a9c437640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-13T07:33:25.195 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:25.194+0000 7f8a9c437640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-13T07:33:25.195 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:25.194+0000 7f8a9c437640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-13T07:33:25.195 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:25.194+0000 7f8a9c437640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-13T07:33:28.037 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-13T07:33:28.068 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-13T07:33:28.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:28.159+0000 7ff5fca48640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-13T07:33:28.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:28.159+0000 7ff5fca48640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-13T07:33:28.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:28.159+0000 7ff5fca48640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-13T07:33:28.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:28.159+0000 7ff5fca48640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-13T07:33:28.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:28.159+0000 7ff5fca48640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-13T07:33:28.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:28.160+0000 7ff5fca48640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-13T07:33:30.998 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-13T07:33:31.029 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-13T07:33:31.124 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:31.123+0000 7f0ca460d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-13T07:33:31.124 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:31.123+0000 7f0ca460d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-13T07:33:31.124 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:31.123+0000 7f0ca460d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-13T07:33:31.124 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:31.123+0000 7f0ca460d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-13T07:33:31.124 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:31.123+0000 7f0ca460d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-13T07:33:31.124 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:31.124+0000 7f0ca460d640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-13T07:33:33.967 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-13T07:33:33.998 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-13T07:33:33.999 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:33.999 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-13T07:33:34.031 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:34.031 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-13T07:33:34.102 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:34.102 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-13T07:33:34.172 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:34.172 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-13T07:33:34.241 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:34.242 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-13T07:33:34.311 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:34.311 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-13T07:33:34.342 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:34.342 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-13T07:33:34.412 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:34.412 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-13T07:33:34.481 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:34.482 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-13T07:33:34.550 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:34.550 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-13T07:33:34.618 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:34.618 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-13T07:33:34.636 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:34.636 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-13T07:33:34.674 INFO:tasks.ceph:Adding keys to all mons... 2024-03-13T07:33:34.674 DEBUG:teuthology.orchestra.run.smithi132:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-13T07:33:34.680 DEBUG:teuthology.orchestra.run.smithi195:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-13T07:33:34.717 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr.y] 2024-03-13T07:33:34.718 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQCxVvFlAGJXKRAAYBfhzzdeOwVIQtBgmL9jYA== 2024-03-13T07:33:34.718 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.0] 2024-03-13T07:33:34.718 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQC1VvFlZUXEBxAAQe50EXscw8jyWM1zzil35w== 2024-03-13T07:33:34.718 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.1] 2024-03-13T07:33:34.718 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQC4VvFl6yEaBhAACBg5P62FJAkYEQamWgkKHw== 2024-03-13T07:33:34.718 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.2] 2024-03-13T07:33:34.718 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQC7VvFlQiPvAxAAP+zn4qWmoP4g/o+ibK/AmQ== 2024-03-13T07:33:34.718 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.3] 2024-03-13T07:33:34.718 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQC+VvFlkZLEARAAHDXI/nSAYoaM9xFmkMS5rA== 2024-03-13T07:33:34.718 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr.x] 2024-03-13T07:33:34.718 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQCxVvFlgrnyLRAACc8OfPEyJWqKKFMsbv6Lpw== 2024-03-13T07:33:34.718 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.4] 2024-03-13T07:33:34.719 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDCVvFlkAIFDhAAB9grHJNvuJYIva5l4Lq6RA== 2024-03-13T07:33:34.719 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.5] 2024-03-13T07:33:34.719 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDFVvFl3aumCxAA26ldIf/AHhjBaCJCSRKjjw== 2024-03-13T07:33:34.719 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.6] 2024-03-13T07:33:34.719 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDIVvFl0NuVCRAAk3bV9X+Vbz5mJu7/NJ7ONw== 2024-03-13T07:33:34.719 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.7] 2024-03-13T07:33:34.719 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDLVvFlrrxvBxAA3MNVQCtbrwi9suByKHCHOA== 2024-03-13T07:33:34.719 INFO:teuthology.orchestra.run.smithi132.stdout:[client.0] 2024-03-13T07:33:34.719 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQCxVvFl3PmoMBAAYmuGQ0B8p2Lk5RY3Fvbo5w== 2024-03-13T07:33:34.719 INFO:teuthology.orchestra.run.smithi132.stdout:[client.1] 2024-03-13T07:33:34.719 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQCxVvFlo6EVNBAAZ5SvVFiKIZILsusfi2r8SQ== 2024-03-13T07:33:34.740 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr.y] 2024-03-13T07:33:34.740 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCxVvFlAGJXKRAAYBfhzzdeOwVIQtBgmL9jYA== 2024-03-13T07:33:34.740 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.0] 2024-03-13T07:33:34.741 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQC1VvFlZUXEBxAAQe50EXscw8jyWM1zzil35w== 2024-03-13T07:33:34.741 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.1] 2024-03-13T07:33:34.741 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQC4VvFl6yEaBhAACBg5P62FJAkYEQamWgkKHw== 2024-03-13T07:33:34.741 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.2] 2024-03-13T07:33:34.741 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQC7VvFlQiPvAxAAP+zn4qWmoP4g/o+ibK/AmQ== 2024-03-13T07:33:34.741 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.3] 2024-03-13T07:33:34.741 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQC+VvFlkZLEARAAHDXI/nSAYoaM9xFmkMS5rA== 2024-03-13T07:33:34.741 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr.x] 2024-03-13T07:33:34.741 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCxVvFlgrnyLRAACc8OfPEyJWqKKFMsbv6Lpw== 2024-03-13T07:33:34.741 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.4] 2024-03-13T07:33:34.741 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQDCVvFlkAIFDhAAB9grHJNvuJYIva5l4Lq6RA== 2024-03-13T07:33:34.741 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.5] 2024-03-13T07:33:34.741 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQDFVvFl3aumCxAA26ldIf/AHhjBaCJCSRKjjw== 2024-03-13T07:33:34.742 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.6] 2024-03-13T07:33:34.742 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQDIVvFl0NuVCRAAk3bV9X+Vbz5mJu7/NJ7ONw== 2024-03-13T07:33:34.742 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.7] 2024-03-13T07:33:34.742 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQDLVvFlrrxvBxAA3MNVQCtbrwi9suByKHCHOA== 2024-03-13T07:33:34.742 INFO:teuthology.orchestra.run.smithi195.stdout:[client.0] 2024-03-13T07:33:34.742 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCxVvFl3PmoMBAAYmuGQ0B8p2Lk5RY3Fvbo5w== 2024-03-13T07:33:34.742 INFO:teuthology.orchestra.run.smithi195.stdout:[client.1] 2024-03-13T07:33:34.742 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCxVvFlo6EVNBAAZ5SvVFiKIZILsusfi2r8SQ== 2024-03-13T07:33:34.744 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-13T07:33:34.764 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-13T07:33:34.838 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T07:33:34.840 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T07:33:34.893 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T07:33:34.897 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T07:33:34.949 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T07:33:34.951 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T07:33:35.003 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T07:33:35.005 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T07:33:35.057 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-13T07:33:35.059 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-13T07:33:35.112 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T07:33:35.114 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T07:33:35.166 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T07:33:35.169 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T07:33:35.220 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T07:33:35.222 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T07:33:35.274 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T07:33:35.277 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T07:33:35.328 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-13T07:33:35.331 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-13T07:33:35.385 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-13T07:33:35.387 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-13T07:33:35.441 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-13T07:33:35.442 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-13T07:33:35.468 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-13T07:33:35.565 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-13T07:33:35.591 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-13T07:33:35.658 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-13T07:33:35.754 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-13T07:33:35.781 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-13T07:33:35.809 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-13T07:33:35.906 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-13T07:33:35.934 DEBUG:teuthology.orchestra.run.smithi132:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-13T07:33:35.936 DEBUG:teuthology.orchestra.run.smithi195:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-13T07:33:35.989 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-13T07:33:35.989 DEBUG:tasks.ceph_manager:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-03-13T07:33:35.989 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-13T07:33:35.989 DEBUG:teuthology.orchestra.run.smithi132:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2024-03-13T07:33:35.992 INFO:tasks.ceph.mon.a:Started 2024-03-13T07:33:35.992 DEBUG:tasks.ceph_manager:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-03-13T07:33:35.992 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-13T07:33:35.992 DEBUG:teuthology.orchestra.run.smithi132:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2024-03-13T07:33:35.994 INFO:tasks.ceph.mon.c:Started 2024-03-13T07:33:35.994 DEBUG:tasks.ceph_manager:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-03-13T07:33:35.994 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-13T07:33:35.994 DEBUG:teuthology.orchestra.run.smithi195:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2024-03-13T07:33:36.031 INFO:tasks.ceph.mon.b:Started 2024-03-13T07:33:36.032 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-13T07:33:36.032 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-13T07:33:36.032 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2024-03-13T07:33:36.034 INFO:tasks.ceph.mgr.y:Started 2024-03-13T07:33:36.034 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-13T07:33:36.035 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2024-03-13T07:33:36.036 INFO:tasks.ceph.mgr.x:Started 2024-03-13T07:33:36.037 INFO:tasks.ceph:Setting crush tunables to default 2024-03-13T07:33:36.037 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd crush tunables default 2024-03-13T07:33:36.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.138+0000 7fb08abcc640 10 monclient: get_monmap_and_config 2024-03-13T07:33:36.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.138+0000 7fb08abcc640 10 monclient: build_initial_monmap 2024-03-13T07:33:36.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.138+0000 7fb08abcc640 10 monclient: monmap: 2024-03-13T07:33:36.139 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:33:36.139 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:36.139 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:36.139154+0000 2024-03-13T07:33:36.139 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:36.139154+0000 2024-03-13T07:33:36.139 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:36.139 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:36.139 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:33:36.139 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:33:36.139 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:33:36.139 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:33:36.139 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:33:36.139 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:33:36.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 1 Processor -- start 2024-03-13T07:33:36.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 1 -- start start 2024-03-13T07:33:36.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 10 monclient: init 2024-03-13T07:33:36.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:36.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 10 monclient: _add_conns ranks=[2,0,3,4,1] 2024-03-13T07:33:36.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0840b5320 0x7fb0840b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:36.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 10 monclient(hunting): picked mon.noname-b con 0x7fb0840b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:36.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb0840b1ac0 addr v1:172.21.15.132:6789/0 2024-03-13T07:33:36.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb08413fd40 0x7fb0840b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:36.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 10 monclient(hunting): picked mon.noname-a con 0x7fb08413fd40 addr v2:172.21.15.132:3300/0 2024-03-13T07:33:36.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0840b1150 con 0x7fb0840b1ac0 2024-03-13T07:33:36.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:36.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0840b5700 con 0x7fb08413fd40 2024-03-13T07:33:36.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:36.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0840b13c0 con 0x7fb0840b5320 2024-03-13T07:33:36.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:36.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb08abcc640 10 monclient(hunting): authenticate will time out at 1243.547919s 2024-03-13T07:33:36.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb083fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb0840b1ac0 legacy=0x7fb08413f630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-03-13T07:33:36.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb089169640 1 -- >> v2:172.21.15.132:3300/0 conn(0x7fb08413fd40 msgr2=0x7fb0840b1f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.132:3300/0 2024-03-13T07:33:36.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb089169640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb08413fd40 0x7fb0840b1f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T07:33:36.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb088968640 1 -- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0840b5320 msgr2=0x7fb0840b0220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.132:3301/0 2024-03-13T07:33:36.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.139+0000 7fb088968640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0840b5320 0x7fb0840b0220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T07:33:36.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.340+0000 7fb089169640 1 -- >> v2:172.21.15.132:3300/0 conn(0x7fb08413fd40 msgr2=0x7fb0840b1f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.132:3300/0 2024-03-13T07:33:36.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.340+0000 7fb083fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb0840b1ac0 legacy=0x7fb08413f630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-03-13T07:33:36.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.340+0000 7fb089169640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb08413fd40 0x7fb0840b1f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-13T07:33:36.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.340+0000 7fb088968640 1 -- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0840b5320 msgr2=0x7fb0840b0220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.132:3301/0 2024-03-13T07:33:36.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.340+0000 7fb088968640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0840b5320 0x7fb0840b0220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-13T07:33:36.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.741+0000 7fb088968640 1 -- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0840b5320 msgr2=0x7fb0840b0220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.132:3301/0 2024-03-13T07:33:36.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.741+0000 7fb083fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb0840b1ac0 legacy=0x7fb08413f630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-03-13T07:33:36.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.741+0000 7fb089169640 1 -- >> v2:172.21.15.132:3300/0 conn(0x7fb08413fd40 msgr2=0x7fb0840b1f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.132:3300/0 2024-03-13T07:33:36.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.741+0000 7fb088968640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0840b5320 0x7fb0840b0220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-13T07:33:36.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:36.741+0000 7fb089169640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb08413fd40 0x7fb0840b1f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-13T07:33:37.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:37.542+0000 7fb089169640 1 -- >> v2:172.21.15.132:3300/0 conn(0x7fb08413fd40 msgr2=0x7fb0840b1f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.132:3300/0 2024-03-13T07:33:37.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:37.542+0000 7fb083fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb0840b1ac0 legacy=0x7fb08413f630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-03-13T07:33:37.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:37.542+0000 7fb088968640 1 -- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0840b5320 msgr2=0x7fb0840b0220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.132:3301/0 2024-03-13T07:33:37.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:37.542+0000 7fb089169640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb08413fd40 0x7fb0840b1f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-13T07:33:37.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:37.542+0000 7fb088968640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0840b5320 0x7fb0840b0220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-13T07:33:39.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 10 monclient(hunting): tick 2024-03-13T07:33:39.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 10 monclient(hunting): _check_auth_tickets 2024-03-13T07:33:39.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 1 monclient(hunting): continuing hunt 2024-03-13T07:33:39.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 10 monclient(hunting): _reopen_session rank -1 2024-03-13T07:33:39.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 1 -- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0840b5320 msgr2=0x7fb0840b0220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-13T07:33:39.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0840b5320 0x7fb0840b0220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:39.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 1 -- >> v2:172.21.15.132:3300/0 conn(0x7fb08413fd40 msgr2=0x7fb0840b1f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-13T07:33:39.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb08413fd40 0x7fb0840b1f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:39.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb0840b1ac0 legacy=0x7fb08413f630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-13T07:33:39.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 10 monclient: _add_conns ranks=[4,1] 2024-03-13T07:33:39.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb0680036c0 0x7fb068005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:39.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 10 monclient(hunting): picked mon.noname-c con 0x7fb0680036c0 addr v2:172.21.15.195:3300/0 2024-03-13T07:33:39.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb068006230 addr v1:172.21.15.195:6789/0 2024-03-13T07:33:39.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0840b16a0 con 0x7fb068006230 2024-03-13T07:33:39.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:39.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0840b5700 con 0x7fb0680036c0 2024-03-13T07:33:39.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.139+0000 7fb0827fc640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:39.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.140+0000 7fb088968640 1 -- >> v2:172.21.15.195:3300/0 conn(0x7fb0680036c0 msgr2=0x7fb068005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.195:3300/0 2024-03-13T07:33:39.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.140+0000 7fb083fff640 1 -- >> v1:172.21.15.195:6789/0 conn(0x7fb068006230 legacy=0x7fb0680086f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.195:6789/0 2024-03-13T07:33:39.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.140+0000 7fb088968640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb0680036c0 0x7fb068005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T07:33:39.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.341+0000 7fb088968640 1 -- >> v2:172.21.15.195:3300/0 conn(0x7fb0680036c0 msgr2=0x7fb068005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.195:3300/0 2024-03-13T07:33:39.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.341+0000 7fb083fff640 1 -- >> v1:172.21.15.195:6789/0 conn(0x7fb068006230 legacy=0x7fb0680086f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.195:6789/0 2024-03-13T07:33:39.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.341+0000 7fb088968640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb0680036c0 0x7fb068005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-13T07:33:39.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.742+0000 7fb088968640 1 -- >> v2:172.21.15.195:3300/0 conn(0x7fb0680036c0 msgr2=0x7fb068005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.195:3300/0 2024-03-13T07:33:39.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.742+0000 7fb083fff640 1 -- >> v1:172.21.15.195:6789/0 conn(0x7fb068006230 legacy=0x7fb0680086f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.195:6789/0 2024-03-13T07:33:39.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:39.742+0000 7fb088968640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb0680036c0 0x7fb068005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-13T07:33:40.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:40.543+0000 7fb088968640 1 -- >> v2:172.21.15.195:3300/0 conn(0x7fb0680036c0 msgr2=0x7fb068005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.195:3300/0 2024-03-13T07:33:40.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:40.543+0000 7fb083fff640 1 -- >> v1:172.21.15.195:6789/0 conn(0x7fb068006230 legacy=0x7fb0680086f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.195:6789/0 2024-03-13T07:33:40.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:40.543+0000 7fb088968640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb0680036c0 0x7fb068005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-13T07:33:42.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 10 monclient(hunting): tick 2024-03-13T07:33:42.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 10 monclient(hunting): _check_auth_tickets 2024-03-13T07:33:42.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 1 monclient(hunting): continuing hunt 2024-03-13T07:33:42.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 10 monclient(hunting): _reopen_session rank -1 2024-03-13T07:33:42.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 1 -- >> v2:172.21.15.195:3300/0 conn(0x7fb0680036c0 msgr2=0x7fb068005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-13T07:33:42.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb0680036c0 0x7fb068005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:42.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 1 -- >> v1:172.21.15.195:6789/0 conn(0x7fb068006230 legacy=0x7fb0680086f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-13T07:33:42.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb089169640 1 -- reap_dead start 2024-03-13T07:33:42.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 10 monclient: _add_conns ranks=[0,2,3,1,4] 2024-03-13T07:33:42.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb068009f80 addr v1:172.21.15.132:6789/0 2024-03-13T07:33:42.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800ab10 0x7fb068005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:42.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 10 monclient(hunting): picked mon.noname-b con 0x7fb06800ab10 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:42.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 0x7fb06800d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:42.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 10 monclient(hunting): picked mon.noname-a con 0x7fb068008790 addr v2:172.21.15.132:3300/0 2024-03-13T07:33:42.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0840b1150 con 0x7fb068009f80 2024-03-13T07:33:42.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:42.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0840b5700 con 0x7fb068008790 2024-03-13T07:33:42.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:42.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0840b13c0 con 0x7fb06800ab10 2024-03-13T07:33:42.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb0827fc640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:42.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb089169640 1 -- >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 msgr2=0x7fb06800d190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.132:3300/0 2024-03-13T07:33:42.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb088968640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb068009f80 legacy=0x7fb06800a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-03-13T07:33:42.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb083fff640 1 -- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800ab10 msgr2=0x7fb068005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.132:3301/0 2024-03-13T07:33:42.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb089169640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 0x7fb06800d190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T07:33:42.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.140+0000 7fb083fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800ab10 0x7fb068005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T07:33:42.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.341+0000 7fb089169640 1 -- >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 msgr2=0x7fb06800d190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.132:3300/0 2024-03-13T07:33:42.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.341+0000 7fb088968640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb068009f80 legacy=0x7fb06800a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-03-13T07:33:42.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.341+0000 7fb083fff640 1 -- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800ab10 msgr2=0x7fb068005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.132:3301/0 2024-03-13T07:33:42.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.341+0000 7fb089169640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 0x7fb06800d190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-13T07:33:42.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.341+0000 7fb083fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800ab10 0x7fb068005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-13T07:33:42.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.742+0000 7fb083fff640 1 -- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800ab10 msgr2=0x7fb068005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.132:3301/0 2024-03-13T07:33:42.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.742+0000 7fb089169640 1 -- >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 msgr2=0x7fb06800d190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.132:3300/0 2024-03-13T07:33:42.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.742+0000 7fb088968640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb068009f80 legacy=0x7fb06800a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-03-13T07:33:42.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.742+0000 7fb083fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800ab10 0x7fb068005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-13T07:33:42.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:42.742+0000 7fb089169640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 0x7fb06800d190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-13T07:33:43.051 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:00:06.921 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:33:43.359 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:00:07.229 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:33:43.371 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:00:07.261 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:33:43.372 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:00:07.262 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:33:43.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:43.543+0000 7fb089169640 1 -- >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 msgr2=0x7fb06800d190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.132:3300/0 2024-03-13T07:33:43.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:43.543+0000 7fb088968640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb068009f80 legacy=0x7fb06800a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-03-13T07:33:43.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:43.543+0000 7fb083fff640 1 -- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800ab10 msgr2=0x7fb068005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.132:3301/0 2024-03-13T07:33:43.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:43.543+0000 7fb089169640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 0x7fb06800d190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-13T07:33:43.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:43.543+0000 7fb083fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800ab10 0x7fb068005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-13T07:33:43.683 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:00:07.573 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:33:43.684 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:00:07.575 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:33:45.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 10 monclient(hunting): tick 2024-03-13T07:33:45.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 10 monclient(hunting): _check_auth_tickets 2024-03-13T07:33:45.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 1 monclient(hunting): continuing hunt 2024-03-13T07:33:45.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 10 monclient(hunting): _reopen_session rank -1 2024-03-13T07:33:45.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 1 -- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800ab10 msgr2=0x7fb068005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-13T07:33:45.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800ab10 0x7fb068005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:45.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 1 -- >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 msgr2=0x7fb06800d190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-13T07:33:45.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 0x7fb06800d190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:45.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb068009f80 legacy=0x7fb06800a360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-13T07:33:45.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 10 monclient: _add_conns ranks=[4,1] 2024-03-13T07:33:45.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 0x7fb06800db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:45.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 10 monclient(hunting): picked mon.noname-c con 0x7fb06800d700 addr v2:172.21.15.195:3300/0 2024-03-13T07:33:45.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb06800e1e0 addr v1:172.21.15.195:6789/0 2024-03-13T07:33:45.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0840b16a0 con 0x7fb06800e1e0 2024-03-13T07:33:45.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:45.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0840b5700 con 0x7fb06800d700 2024-03-13T07:33:45.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.140+0000 7fb0827fc640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:45.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.246+0000 7fb088968640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 0x7fb06800db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:45.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.247+0000 7fb083fff640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fb06800e1e0 0x7fb0680127b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.132:40390/0 (socket says 172.21.15.132:40390) 2024-03-13T07:33:45.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.247+0000 7fb083fff640 1 -- 172.21.15.132:0/3476021786 learned_addr learned my addr 172.21.15.132:0/3476021786 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:33:45.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.247+0000 7fb088968640 10 monclient(hunting): get_auth_request con 0x7fb06800d700 auth_method 0 2024-03-13T07:33:45.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.247+0000 7fb088968640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:45.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.247+0000 7fb088968640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:45.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.247+0000 7fb088968640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:45.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.283+0000 7fb088968640 1 -- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 msgr2=0x7fb06800db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-13T07:33:45.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.283+0000 7fb088968640 1 -- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 msgr2=0x7fb06800db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T07:33:45.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.283+0000 7fb088968640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 0x7fb06800db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-13T07:33:45.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.283+0000 7fb088968640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 0x7fb06800db10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T07:33:45.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.692+0000 7fb088968640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 0x7fb06800db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:45.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.693+0000 7fb088968640 10 monclient(hunting): get_auth_request con 0x7fb06800d700 auth_method 0 2024-03-13T07:33:45.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.693+0000 7fb088968640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:45.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.693+0000 7fb088968640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:45.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.693+0000 7fb088968640 10 monclient(hunting): _init_auth already have auth, reseting 2024-03-13T07:33:45.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.693+0000 7fb088968640 1 -- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 msgr2=0x7fb06800db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-13T07:33:45.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.693+0000 7fb088968640 1 -- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 msgr2=0x7fb06800db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T07:33:45.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.693+0000 7fb088968640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 0x7fb06800db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-13T07:33:45.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:45.693+0000 7fb088968640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 0x7fb06800db10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-13T07:33:46.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.166+0000 7fb088968640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 0x7fb06800db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:46.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.166+0000 7fb088968640 10 monclient(hunting): get_auth_request con 0x7fb06800d700 auth_method 0 2024-03-13T07:33:46.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.166+0000 7fb088968640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:46.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.166+0000 7fb088968640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:46.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.166+0000 7fb088968640 10 monclient(hunting): _init_auth already have auth, reseting 2024-03-13T07:33:46.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.167+0000 7fb088968640 1 -- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 msgr2=0x7fb06800db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-13T07:33:46.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.167+0000 7fb088968640 1 -- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 msgr2=0x7fb06800db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T07:33:46.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.167+0000 7fb088968640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 0x7fb06800db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-13T07:33:46.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.167+0000 7fb088968640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 0x7fb06800db10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-13T07:33:46.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.969+0000 7fb088968640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 0x7fb06800db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:46.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.969+0000 7fb088968640 10 monclient(hunting): get_auth_request con 0x7fb06800d700 auth_method 0 2024-03-13T07:33:46.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.969+0000 7fb088968640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:46.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.969+0000 7fb088968640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:46.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.969+0000 7fb088968640 10 monclient(hunting): _init_auth already have auth, reseting 2024-03-13T07:33:46.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.970+0000 7fb088968640 1 -- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 msgr2=0x7fb06800db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-13T07:33:46.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.970+0000 7fb088968640 1 -- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 msgr2=0x7fb06800db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T07:33:46.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.970+0000 7fb088968640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 0x7fb06800db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-13T07:33:46.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:46.970+0000 7fb088968640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 0x7fb06800db10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-13T07:33:48.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 10 monclient(hunting): tick 2024-03-13T07:33:48.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 10 monclient(hunting): _check_auth_tickets 2024-03-13T07:33:48.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 1 monclient(hunting): continuing hunt 2024-03-13T07:33:48.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 10 monclient(hunting): _reopen_session rank -1 2024-03-13T07:33:48.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 1 -- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 msgr2=0x7fb06800db10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-13T07:33:48.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.195:3300/0 conn(0x7fb06800d700 0x7fb06800db10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:48.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 1 -- 172.21.15.132:0/3476021786 >> v1:172.21.15.195:6789/0 conn(0x7fb06800e1e0 legacy=0x7fb0680127b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:48.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 10 monclient: _add_conns ranks=[3,2,1,0,4] 2024-03-13T07:33:48.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb089169640 1 -- 172.21.15.132:0/3476021786 reap_dead start 2024-03-13T07:33:48.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 0x7fb06800d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:48.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 10 monclient(hunting): picked mon.noname-a con 0x7fb068008790 addr v2:172.21.15.132:3300/0 2024-03-13T07:33:48.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 1 --2- 172.21.15.132:0/3476021786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 0x7fb068009f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:48.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 10 monclient(hunting): picked mon.noname-b con 0x7fb06800db10 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:48.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb06800a4c0 addr v1:172.21.15.195:6789/0 2024-03-13T07:33:48.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 1 -- 172.21.15.132:0/3476021786 --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0840b1150 con 0x7fb06800a4c0 2024-03-13T07:33:48.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:48.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 1 -- 172.21.15.132:0/3476021786 --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0840b5700 con 0x7fb068008790 2024-03-13T07:33:48.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:48.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 1 -- 172.21.15.132:0/3476021786 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0840b13c0 con 0x7fb06800db10 2024-03-13T07:33:48.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.140+0000 7fb0827fc640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:48.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.141+0000 7fb083fff640 1 --2- 172.21.15.132:0/3476021786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 0x7fb068009f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:48.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.141+0000 7fb083fff640 10 monclient(hunting): get_auth_request con 0x7fb06800db10 auth_method 0 2024-03-13T07:33:48.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.141+0000 7fb083fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:48.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.141+0000 7fb083fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:48.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.141+0000 7fb083fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:48.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.147+0000 7fb083fff640 1 -- 172.21.15.132:0/3476021786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 msgr2=0x7fb068009f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-13T07:33:48.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.147+0000 7fb083fff640 1 -- 172.21.15.132:0/3476021786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 msgr2=0x7fb068009f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T07:33:48.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.147+0000 7fb083fff640 1 --2- 172.21.15.132:0/3476021786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 0x7fb068009f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-13T07:33:48.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.147+0000 7fb083fff640 1 --2- 172.21.15.132:0/3476021786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 0x7fb068009f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T07:33:48.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.211+0000 7fb088968640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 0x7fb06800d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:48.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.211+0000 7fb088968640 10 monclient(hunting): get_auth_request con 0x7fb068008790 auth_method 0 2024-03-13T07:33:48.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.211+0000 7fb088968640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:48.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.211+0000 7fb088968640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:48.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.211+0000 7fb088968640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:48.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.213+0000 7fb088968640 1 -- 172.21.15.132:0/3476021786 >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 msgr2=0x7fb06800d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-13T07:33:48.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.213+0000 7fb088968640 1 -- 172.21.15.132:0/3476021786 >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 msgr2=0x7fb06800d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T07:33:48.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.213+0000 7fb088968640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 0x7fb06800d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-13T07:33:48.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.213+0000 7fb088968640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 0x7fb06800d190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T07:33:48.315 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.315+0000 7fb0837fe640 1 -- 172.21.15.132:0/3476021786 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638811666 0 0) 0x7fb0840b1150 con 0x7fb06800a4c0 2024-03-13T07:33:48.315 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.315+0000 7fb0837fe640 10 monclient(hunting): _negotiate 2024-03-13T07:33:48.315 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.315+0000 7fb0837fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:48.315 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.315+0000 7fb0837fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:48.315 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.315+0000 7fb0837fe640 10 monclient(hunting): my global_id is 4102 2024-03-13T07:33:48.315 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.315+0000 7fb0837fe640 1 -- 172.21.15.132:0/3476021786 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb074003520 con 0x7fb06800a4c0 2024-03-13T07:33:48.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.356+0000 7fb083fff640 1 --2- 172.21.15.132:0/3476021786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 0x7fb068009f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:48.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.416+0000 7fb083fff640 10 monclient(hunting): get_auth_request con 0x7fb06800db10 auth_method 0 2024-03-13T07:33:48.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.416+0000 7fb083fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:48.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.416+0000 7fb083fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:48.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.416+0000 7fb083fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-03-13T07:33:48.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.430+0000 7fb083fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:48.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.430+0000 7fb083fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:48.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.430+0000 7fb083fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:48.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.463+0000 7fb083fff640 10 monclient(hunting): handle_auth_done global_id 4100 payload 274 2024-03-13T07:33:48.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.463+0000 7fb083fff640 1 -- 172.21.15.132:0/3476021786 >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 msgr2=0x7fb06800d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:33:48.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.463+0000 7fb083fff640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 0x7fb06800d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:48.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.463+0000 7fb083fff640 1 -- 172.21.15.132:0/3476021786 >> v1:172.21.15.195:6789/0 conn(0x7fb06800a4c0 legacy=0x7fb068013f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:48.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.463+0000 7fb083fff640 10 monclient: _finish_hunting 0 2024-03-13T07:33:48.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.463+0000 7fb083fff640 1 monclient: found mon.noname-b 2024-03-13T07:33:48.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.463+0000 7fb083fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:48.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.463+0000 7fb083fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T07:33:48.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.463+0000 7fb083fff640 1 -- 172.21.15.132:0/3476021786 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb068005b40 con 0x7fb06800db10 2024-03-13T07:33:48.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.468+0000 7fb083fff640 1 --2- 172.21.15.132:0/3476021786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 0x7fb068009f80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb070007fd0 tx=0x7fb07002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66abd70c9cb940ac server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:48.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb0837fe640 1 -- 172.21.15.132:0/3476021786 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb070005050 con 0x7fb06800db10 2024-03-13T07:33:48.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb0837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:48.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb0837fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:33:48.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb0837fe640 10 monclient: dump: 2024-03-13T07:33:48.507 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:33:48.507 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:48.507 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:48.507 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:48.508 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:48.508 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:48.508 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:48.508 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:48.508 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:48.508 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:33:48.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb0837fe640 10 monclient: _finish_auth 0 2024-03-13T07:33:48.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb0837fe640 10 monclient: _check_auth_tickets 2024-03-13T07:33:48.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb0837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:48.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb0837fe640 1 -- 172.21.15.132:0/3476021786 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb070036070 con 0x7fb06800db10 2024-03-13T07:33:48.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb08abcc640 5 monclient: authenticate success, global_id 4100 2024-03-13T07:33:48.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb08abcc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:33:48.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb0837fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:48.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb08abcc640 10 monclient: get_monmap_and_config success 2024-03-13T07:33:48.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb08abcc640 4 set_mon_vals no callback set 2024-03-13T07:33:48.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb0837fe640 1 -- 172.21.15.132:0/3476021786 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb070005430 con 0x7fb06800db10 2024-03-13T07:33:48.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb0837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:48.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb0837fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:33:48.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.507+0000 7fb0837fe640 10 monclient: dump: 2024-03-13T07:33:48.509 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:33:48.509 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:48.509 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:48.509 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:48.509 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:48.509 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:48.509 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:48.509 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:48.509 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:48.509 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:33:48.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.508+0000 7fb08abcc640 10 monclient: shutdown 2024-03-13T07:33:48.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.508+0000 7fb08abcc640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:48.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.508+0000 7fb08abcc640 1 -- 172.21.15.132:0/3476021786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 msgr2=0x7fb068009f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:48.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.508+0000 7fb08abcc640 1 --2- 172.21.15.132:0/3476021786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 0x7fb068009f80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb070007fd0 tx=0x7fb07002b8d0 comp rx=0 tx=0).stop 2024-03-13T07:33:48.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.508+0000 7fb08abcc640 1 -- 172.21.15.132:0/3476021786 shutdown_connections 2024-03-13T07:33:48.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.508+0000 7fb08abcc640 1 --2- 172.21.15.132:0/3476021786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 0x7fb068009f80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:48.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.508+0000 7fb08abcc640 1 --2- 172.21.15.132:0/3476021786 >> v2:172.21.15.132:3300/0 conn(0x7fb068008790 0x7fb06800d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:48.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.508+0000 7fb08abcc640 1 -- 172.21.15.132:0/3476021786 >> 172.21.15.132:0/3476021786 conn(0x7fb08406d2b0 msgr2=0x7fb08406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:48.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.508+0000 7fb08abcc640 1 -- 172.21.15.132:0/3476021786 shutdown_connections 2024-03-13T07:33:48.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.508+0000 7fb08abcc640 1 -- 172.21.15.132:0/3476021786 wait complete. 2024-03-13T07:33:48.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.509+0000 7fb08abcc640 10 monclient: build_initial_monmap 2024-03-13T07:33:48.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.509+0000 7fb08abcc640 10 monclient: monmap: 2024-03-13T07:33:48.514 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:33:48.514 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:33:48.514 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:33:48.514 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:33:48.515 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:48.515 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:48.515 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:33:48.515 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:33:48.515 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:33:48.515 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:33:48.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.509+0000 7fb08abcc640 1 librados: starting msgr at 2024-03-13T07:33:48.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.509+0000 7fb08abcc640 1 librados: starting objecter 2024-03-13T07:33:48.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.509+0000 7fb08abcc640 1 Processor -- start 2024-03-13T07:33:48.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.509+0000 7fb08abcc640 1 -- start start 2024-03-13T07:33:48.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.509+0000 7fb08abcc640 1 librados: setting wanted keys 2024-03-13T07:33:48.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.509+0000 7fb08abcc640 1 librados: calling monclient init 2024-03-13T07:33:48.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.509+0000 7fb08abcc640 10 monclient: init 2024-03-13T07:33:48.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.510+0000 7fb08abcc640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:48.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.510+0000 7fb08abcc640 10 monclient: _add_conns ranks=[0,1,2] 2024-03-13T07:33:48.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.510+0000 7fb08abcc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb068008790 0x7fb0841dafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:48.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.510+0000 7fb08abcc640 10 monclient(hunting): picked mon.noname-a con 0x7fb068008790 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:33:48.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.510+0000 7fb08abcc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb06800a4c0 0x7fb084140ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:48.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.510+0000 7fb08abcc640 10 monclient(hunting): picked mon.noname-b con 0x7fb06800a4c0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:33:48.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.510+0000 7fb08abcc640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 0x7fb084141b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:48.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.510+0000 7fb08abcc640 10 monclient(hunting): picked mon.noname-c con 0x7fb06800db10 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:48.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.510+0000 7fb08abcc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:48.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.510+0000 7fb08abcc640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0840b4800 con 0x7fb068008790 2024-03-13T07:33:48.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.510+0000 7fb08abcc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:48.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.510+0000 7fb08abcc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0840afd00 con 0x7fb06800a4c0 2024-03-13T07:33:48.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.510+0000 7fb08abcc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:48.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.510+0000 7fb08abcc640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0840b4980 con 0x7fb06800db10 2024-03-13T07:33:48.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.510+0000 7fb08abcc640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:48.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.510+0000 7fb08abcc640 10 monclient(hunting): authenticate will time out at 1255.918507s 2024-03-13T07:33:48.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.522+0000 7fb089169640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 0x7fb084141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:48.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.523+0000 7fb089169640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 0x7fb084141b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:57872/0 (socket says 172.21.15.132:57872) 2024-03-13T07:33:48.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.523+0000 7fb089169640 1 -- 172.21.15.132:0/2793165139 learned_addr learned my addr 172.21.15.132:0/2793165139 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:33:48.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.523+0000 7fb089169640 10 monclient(hunting): get_auth_request con 0x7fb06800db10 auth_method 0 2024-03-13T07:33:48.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.523+0000 7fb089169640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:48.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.523+0000 7fb089169640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:48.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.523+0000 7fb089169640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:48.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.523+0000 7fb089169640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:48.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.523+0000 7fb089169640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:48.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.523+0000 7fb089169640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:48.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.534+0000 7fb089169640 10 monclient(hunting): handle_auth_done global_id 4103 payload 824 2024-03-13T07:33:48.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.534+0000 7fb089169640 1 -- 172.21.15.132:0/2793165139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb06800a4c0 msgr2=0x7fb084140ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:48.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.534+0000 7fb089169640 1 --2- 172.21.15.132:0/2793165139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb06800a4c0 0x7fb084140ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:48.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.534+0000 7fb089169640 1 -- 172.21.15.132:0/2793165139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb068008790 msgr2=0x7fb0841dafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:48.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.534+0000 7fb089169640 1 --2- 172.21.15.132:0/2793165139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb068008790 0x7fb0841dafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:48.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.534+0000 7fb089169640 10 monclient: _finish_hunting 0 2024-03-13T07:33:48.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.534+0000 7fb089169640 1 monclient: found mon.noname-c 2024-03-13T07:33:48.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.534+0000 7fb089169640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:48.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.534+0000 7fb089169640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:33:48.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.534+0000 7fb089169640 1 -- 172.21.15.132:0/2793165139 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb084144c30 con 0x7fb06800db10 2024-03-13T07:33:48.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.536+0000 7fb089169640 1 --2- 172.21.15.132:0/2793165139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 0x7fb084141b00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb068009f30 tx=0x7fb07c009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:48.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.536+0000 7fb0817fa640 10 client.?.objecter ms_handle_connect 0x7fb06800db10 2024-03-13T07:33:48.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.536+0000 7fb0817fa640 10 client.?.objecter resend_mon_ops 2024-03-13T07:33:48.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb0817fa640 1 -- 172.21.15.132:0/2793165139 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c016430 con 0x7fb06800db10 2024-03-13T07:33:48.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb0817fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:48.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb0817fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:33:48.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb0817fa640 10 monclient: dump: 2024-03-13T07:33:48.541 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:33:48.541 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:48.541 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:48.541 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:48.541 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:48.541 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:48.541 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:48.541 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:48.541 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:48.541 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:33:48.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb0817fa640 10 monclient: _finish_auth 0 2024-03-13T07:33:48.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb0817fa640 10 monclient: _check_auth_tickets 2024-03-13T07:33:48.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb0817fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:48.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb0817fa640 1 -- 172.21.15.132:0/2793165139 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07c010030 con 0x7fb06800db10 2024-03-13T07:33:48.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb08abcc640 5 monclient: authenticate success, global_id 4103 2024-03-13T07:33:48.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb0817fa640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:48.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb08abcc640 10 monclient: _renew_subs 2024-03-13T07:33:48.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb0817fa640 1 -- 172.21.15.132:0/2793165139 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c0167f0 con 0x7fb06800db10 2024-03-13T07:33:48.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb08abcc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:48.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb08abcc640 1 -- 172.21.15.132:0/2793165139 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0841db9d0 con 0x7fb06800db10 2024-03-13T07:33:48.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb0817fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:48.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb0817fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:33:48.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb0817fa640 10 monclient: dump: 2024-03-13T07:33:48.542 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:33:48.543 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:48.543 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:48.543 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:48.543 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:48.543 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:48.543 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:48.543 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:48.543 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:48.543 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:33:48.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb082ffd640 4 set_mon_vals no callback set 2024-03-13T07:33:48.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb08abcc640 10 client.4103.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:33:48.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb08abcc640 10 monclient: _renew_subs 2024-03-13T07:33:48.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb08abcc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:48.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb08abcc640 1 -- 172.21.15.132:0/2793165139 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb084142220 con 0x7fb06800db10 2024-03-13T07:33:48.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.540+0000 7fb08abcc640 1 librados: init done 2024-03-13T07:33:48.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.541+0000 7fb08abcc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:48.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.541+0000 7fb08abcc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:48.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.541+0000 7fb08abcc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:48.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.541+0000 7fb08abcc640 1 -- 172.21.15.132:0/2793165139 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0841424b0 con 0x7fb06800db10 2024-03-13T07:33:48.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.737+0000 7fb0817fa640 1 -- 172.21.15.132:0/2793165139 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb07c0207d0 con 0x7fb06800db10 2024-03-13T07:33:48.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.737+0000 7fb0817fa640 1 -- 172.21.15.132:0/2793165139 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb07c01fb50 con 0x7fb06800db10 2024-03-13T07:33:48.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.737+0000 7fb0817fa640 10 client.4103.objecter ms_dispatch 0x7fb084000b90 osd_map(1..1 src has 1..1) v4 2024-03-13T07:33:48.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.737+0000 7fb0817fa640 3 client.4103.objecter handle_osd_map got epochs [1,1] > 0 2024-03-13T07:33:48.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.737+0000 7fb0817fa640 3 client.4103.objecter handle_osd_map decoding full epoch 1 2024-03-13T07:33:48.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.737+0000 7fb0817fa640 20 client.4103.objecter dump_active .. 0 homeless 2024-03-13T07:33:48.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.737+0000 7fb0817fa640 1 -- 172.21.15.132:0/2793165139 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb07c008070 con 0x7fb06800db10 2024-03-13T07:33:48.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.737+0000 7fb0817fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:48.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.737+0000 7fb0817fa640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:33:48.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.806+0000 7fb08abcc640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2024-03-13T07:33:48.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.806+0000 7fb08abcc640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-03-13T07:33:48.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.806+0000 7fb08abcc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:48.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:48.806+0000 7fb08abcc640 1 -- 172.21.15.132:0/2793165139 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fb08404f000 con 0x7fb06800db10 2024-03-13T07:33:49.080 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:49.080+0000 7fd30663e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-13T07:33:49.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.099+0000 7fb0817fa640 1 -- 172.21.15.132:0/2793165139 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb07c01f3e0 con 0x7fb06800db10 2024-03-13T07:33:49.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.099+0000 7fb0817fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-03-13T07:33:49.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.099+0000 7fb0817fa640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2024-03-13T07:33:49.099 INFO:teuthology.orchestra.run.smithi132.stderr:adjusted tunables profile to default 2024-03-13T07:33:49.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.100+0000 7fb08abcc640 10 librados: watch_flush enter 2024-03-13T07:33:49.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.100+0000 7fb08abcc640 10 librados: watch_flush exit 2024-03-13T07:33:49.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.100+0000 7fb08abcc640 20 client.4103.objecter shutdown clearing up homeless session... 2024-03-13T07:33:49.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.100+0000 7fb08abcc640 10 client.4103.objecter successfully canceled tick 2024-03-13T07:33:49.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.100+0000 7fb08abcc640 10 monclient: shutdown 2024-03-13T07:33:49.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.100+0000 7fb08abcc640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:49.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.100+0000 7fb08abcc640 1 -- 172.21.15.132:0/2793165139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 msgr2=0x7fb084141b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:49.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.100+0000 7fb08abcc640 1 --2- 172.21.15.132:0/2793165139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 0x7fb084141b00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb068009f30 tx=0x7fb07c009e30 comp rx=0 tx=0).stop 2024-03-13T07:33:49.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.100+0000 7fb08abcc640 1 -- 172.21.15.132:0/2793165139 shutdown_connections 2024-03-13T07:33:49.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.100+0000 7fb08abcc640 1 --2- 172.21.15.132:0/2793165139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb06800db10 0x7fb084141b00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:49.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.100+0000 7fb08abcc640 1 --2- 172.21.15.132:0/2793165139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb06800a4c0 0x7fb084140ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:49.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.100+0000 7fb08abcc640 1 --2- 172.21.15.132:0/2793165139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb068008790 0x7fb0841dafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:49.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.100+0000 7fb08abcc640 1 -- 172.21.15.132:0/2793165139 >> 172.21.15.132:0/2793165139 conn(0x7fb08406d2b0 msgr2=0x7fb084074530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:49.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.100+0000 7fb08abcc640 1 -- 172.21.15.132:0/2793165139 shutdown_connections 2024-03-13T07:33:49.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.100+0000 7fb08abcc640 1 -- 172.21.15.132:0/2793165139 wait complete. 2024-03-13T07:33:49.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:49.101+0000 7fb08abcc640 1 librados: shutdown 2024-03-13T07:33:49.104 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:49.105+0000 7fb6b5844040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-13T07:33:49.115 INFO:tasks.ceph:check_enable_crimson: False 2024-03-13T07:33:49.115 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-13T07:33:49.115 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:49.115 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-13T07:33:49.135 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:49.135+0000 7fd30663e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-13T07:33:49.142 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:49.142 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-13T07:33:49.165 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:49.165+0000 7fb6b5844040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-13T07:33:49.193 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:49.193 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-13T07:33:49.260 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:33:49.260 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-13T07:33:49.327 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:49.327 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-13T07:33:49.355 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:49.355 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-13T07:33:49.421 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:49.422 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-13T07:33:49.488 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-13T07:33:49.488 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-13T07:33:49.557 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 0edb216c-bc1e-4865-aafe-218611f3eb09 0 2024-03-13T07:33:49.679 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:49.679+0000 7fd30663e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-13T07:33:49.697 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.696+0000 7fe7a7ac2640 10 monclient: get_monmap_and_config 2024-03-13T07:33:49.697 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.696+0000 7fe7a7ac2640 10 monclient: build_initial_monmap 2024-03-13T07:33:49.697 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.696+0000 7fe7a7ac2640 10 monclient: monmap: 2024-03-13T07:33:49.697 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 0 2024-03-13T07:33:49.698 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:49.698 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:49.698017+0000 2024-03-13T07:33:49.698 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:49.698017+0000 2024-03-13T07:33:49.698 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:49.698 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:49.698 INFO:teuthology.orchestra.run.smithi195.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:33:49.698 INFO:teuthology.orchestra.run.smithi195.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:33:49.698 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:33:49.698 INFO:teuthology.orchestra.run.smithi195.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:33:49.698 INFO:teuthology.orchestra.run.smithi195.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:33:49.698 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:49.698 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.697+0000 7fe7a7ac2640 1 Processor -- start 2024-03-13T07:33:49.698 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.697+0000 7fe7a7ac2640 1 -- start start 2024-03-13T07:33:49.698 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.697+0000 7fe7a7ac2640 10 monclient: init 2024-03-13T07:33:49.700 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.698+0000 7fe7a7ac2640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:49.700 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.698+0000 7fe7a7ac2640 10 monclient: _add_conns ranks=[4,3,1,0,2] 2024-03-13T07:33:49.700 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.698+0000 7fe7a7ac2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe7a00b5320 0x7fe7a00b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:49.700 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.698+0000 7fe7a7ac2640 10 monclient(hunting): picked mon.noname-c con 0x7fe7a00b5320 addr v2:172.21.15.195:3300/0 2024-03-13T07:33:49.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.698+0000 7fe7a7ac2640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe7a00b1ac0 0x7fe7a00b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:49.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.698+0000 7fe7a7ac2640 10 monclient(hunting): picked mon.noname-a con 0x7fe7a00b1ac0 addr v2:172.21.15.132:3300/0 2024-03-13T07:33:49.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.698+0000 7fe7a7ac2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe7a013f630 addr v1:172.21.15.195:6789/0 2024-03-13T07:33:49.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.698+0000 7fe7a7ac2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a00b1150 con 0x7fe7a013f630 2024-03-13T07:33:49.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.698+0000 7fe7a7ac2640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:49.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.698+0000 7fe7a7ac2640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a00b13c0 con 0x7fe7a00b1ac0 2024-03-13T07:33:49.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.698+0000 7fe7a7ac2640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:49.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.698+0000 7fe7a7ac2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a00b16a0 con 0x7fe7a00b5320 2024-03-13T07:33:49.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.698+0000 7fe7a7ac2640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:49.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.698+0000 7fe7a7ac2640 10 monclient(hunting): authenticate will time out at 1244.036747s 2024-03-13T07:33:49.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.699+0000 7fe7a585e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe7a00b5320 0x7fe7a00b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:49.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.699+0000 7fe7a505d640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe7a00b1ac0 0x7fe7a00b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:49.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.699+0000 7fe7a505d640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe7a00b1ac0 0x7fe7a00b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.195:52560/0 (socket says 172.21.15.195:52560) 2024-03-13T07:33:49.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.699+0000 7fe7a505d640 1 -- 172.21.15.195:0/3064854142 learned_addr learned my addr 172.21.15.195:0/3064854142 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-13T07:33:49.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.699+0000 7fe7a505d640 10 monclient(hunting): get_auth_request con 0x7fe7a00b1ac0 auth_method 0 2024-03-13T07:33:49.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.699+0000 7fe7a505d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:49.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.699+0000 7fe7a505d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:49.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.699+0000 7fe7a505d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:49.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.699+0000 7fe7a585e640 10 monclient(hunting): get_auth_request con 0x7fe7a00b5320 auth_method 0 2024-03-13T07:33:49.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.699+0000 7fe7a585e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:49.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.699+0000 7fe7a585e640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:49.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.699+0000 7fe7a585e640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:49.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.699+0000 7fe7a505d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:49.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.699+0000 7fe7a505d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:49.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.699+0000 7fe7a505d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:49.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.700+0000 7fe7a585e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:49.702 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.700+0000 7fe7a585e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:49.703 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.700+0000 7fe7a585e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:49.703 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.701+0000 7fe7a505d640 10 monclient(hunting): handle_auth_done global_id 4107 payload 274 2024-03-13T07:33:49.703 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.701+0000 7fe7a505d640 1 -- 172.21.15.195:0/3064854142 >> v2:172.21.15.195:3300/0 conn(0x7fe7a00b5320 msgr2=0x7fe7a00b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:33:49.703 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.701+0000 7fe7a505d640 1 --2- 172.21.15.195:0/3064854142 >> v2:172.21.15.195:3300/0 conn(0x7fe7a00b5320 0x7fe7a00b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:49.703 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.701+0000 7fe7a505d640 1 -- 172.21.15.195:0/3064854142 >> v1:172.21.15.195:6789/0 conn(0x7fe7a013f630 legacy=0x7fe7a013fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:49.703 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.701+0000 7fe7a505d640 10 monclient: _finish_hunting 0 2024-03-13T07:33:49.704 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.701+0000 7fe7a505d640 1 monclient: found mon.noname-a 2024-03-13T07:33:49.704 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.701+0000 7fe7a505d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:49.704 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.701+0000 7fe7a505d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:33:49.704 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.701+0000 7fe7a505d640 1 -- 172.21.15.195:0/3064854142 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00afa20 con 0x7fe7a00b1ac0 2024-03-13T07:33:49.704 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.703+0000 7fe7a505d640 1 --2- 172.21.15.195:0/3064854142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 0x7fe7a00b1f30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00b90f0 tx=0x7fe79002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae468bfe7a14735 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:49.704 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:49.704+0000 7fb6b5844040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-13T07:33:49.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.707+0000 7fe7a485c640 1 -- 172.21.15.195:0/3064854142 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79002e070 con 0x7fe7a00b1ac0 2024-03-13T07:33:49.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.707+0000 7fe7a485c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:49.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a485c640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:33:49.708 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a485c640 10 monclient: dump: 2024-03-13T07:33:49.708 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:49.708 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:49.708 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:49.708 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:49.708 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:49.708 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:49.709 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:49.709 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:49.709 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:49.709 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:49.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a485c640 10 monclient: _finish_auth 0 2024-03-13T07:33:49.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a485c640 10 monclient: _check_auth_tickets 2024-03-13T07:33:49.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a485c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:49.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a485c640 1 -- 172.21.15.195:0/3064854142 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe790037070 con 0x7fe7a00b1ac0 2024-03-13T07:33:49.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a485c640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:49.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a485c640 1 -- 172.21.15.195:0/3064854142 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790031a00 con 0x7fe7a00b1ac0 2024-03-13T07:33:49.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a7ac2640 5 monclient: authenticate success, global_id 4107 2024-03-13T07:33:49.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a7ac2640 10 monclient: get_monmap_and_config success 2024-03-13T07:33:49.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a7ac2640 4 set_mon_vals no callback set 2024-03-13T07:33:49.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a485c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:49.709 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a485c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:33:49.710 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a485c640 10 monclient: dump: 2024-03-13T07:33:49.710 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:49.710 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:49.710 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:49.710 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:49.710 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:49.710 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:49.710 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:49.710 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:49.710 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:49.710 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:49.714 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a7ac2640 10 monclient: shutdown 2024-03-13T07:33:49.714 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a7ac2640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:49.714 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a7ac2640 1 -- 172.21.15.195:0/3064854142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 msgr2=0x7fe7a00b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:49.714 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.708+0000 7fe7a7ac2640 1 --2- 172.21.15.195:0/3064854142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 0x7fe7a00b1f30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00b90f0 tx=0x7fe79002b880 comp rx=0 tx=0).stop 2024-03-13T07:33:49.714 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.709+0000 7fe7a7ac2640 1 -- 172.21.15.195:0/3064854142 shutdown_connections 2024-03-13T07:33:49.714 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.709+0000 7fe7a7ac2640 1 --2- 172.21.15.195:0/3064854142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 0x7fe7a00b1f30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:49.714 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.709+0000 7fe7a7ac2640 1 --2- 172.21.15.195:0/3064854142 >> v2:172.21.15.195:3300/0 conn(0x7fe7a00b5320 0x7fe7a00b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:49.715 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.709+0000 7fe7a7ac2640 1 -- 172.21.15.195:0/3064854142 >> 172.21.15.195:0/3064854142 conn(0x7fe7a006d2b0 msgr2=0x7fe7a006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:49.715 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.709+0000 7fe7a7ac2640 1 -- 172.21.15.195:0/3064854142 shutdown_connections 2024-03-13T07:33:49.715 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.709+0000 7fe7a7ac2640 1 -- 172.21.15.195:0/3064854142 wait complete. 2024-03-13T07:33:49.715 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.709+0000 7fe7a7ac2640 10 monclient: build_initial_monmap 2024-03-13T07:33:49.715 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.709+0000 7fe7a7ac2640 10 monclient: monmap: 2024-03-13T07:33:49.715 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 0 2024-03-13T07:33:49.715 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:33:49.715 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 0.000000 2024-03-13T07:33:49.715 INFO:teuthology.orchestra.run.smithi195.stderr:created 0.000000 2024-03-13T07:33:49.715 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:49.715 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:49.715 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:33:49.715 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:33:49.715 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:33:49.715 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:49.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 1 librados: starting msgr at 2024-03-13T07:33:49.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 1 librados: starting objecter 2024-03-13T07:33:49.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 1 Processor -- start 2024-03-13T07:33:49.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 1 -- start start 2024-03-13T07:33:49.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 1 librados: setting wanted keys 2024-03-13T07:33:49.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 1 librados: calling monclient init 2024-03-13T07:33:49.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 10 monclient: init 2024-03-13T07:33:49.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:49.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T07:33:49.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 0x7fe7a013b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:49.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 10 monclient(hunting): picked mon.noname-a con 0x7fe7a00b1ac0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:33:49.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7a00b5320 0x7fe7a0141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:49.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 10 monclient(hunting): picked mon.noname-c con 0x7fe7a00b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:49.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7a013f630 0x7fe7a0142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:49.716 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 10 monclient(hunting): picked mon.noname-b con 0x7fe7a013f630 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:33:49.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:49.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00b4980 con 0x7fe7a00b1ac0 2024-03-13T07:33:49.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:49.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00b4800 con 0x7fe7a013f630 2024-03-13T07:33:49.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:49.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a014cef0 con 0x7fe7a00b5320 2024-03-13T07:33:49.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:49.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.710+0000 7fe7a7ac2640 10 monclient(hunting): authenticate will time out at 1244.049206s 2024-03-13T07:33:49.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.711+0000 7fe7a605f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7a013f630 0x7fe7a0142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:49.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.711+0000 7fe7a585e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 0x7fe7a013b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:49.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.711+0000 7fe7a505d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7a00b5320 0x7fe7a0141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:49.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.711+0000 7fe7a605f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7a013f630 0x7fe7a0142b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:49804/0 (socket says 172.21.15.195:49804) 2024-03-13T07:33:49.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.711+0000 7fe7a605f640 1 -- 172.21.15.195:0/4156414818 learned_addr learned my addr 172.21.15.195:0/4156414818 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-13T07:33:49.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.711+0000 7fe7a605f640 10 monclient(hunting): get_auth_request con 0x7fe7a013f630 auth_method 0 2024-03-13T07:33:49.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.711+0000 7fe7a605f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:49.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.711+0000 7fe7a605f640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:49.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.711+0000 7fe7a605f640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:49.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.711+0000 7fe7a585e640 10 monclient(hunting): get_auth_request con 0x7fe7a00b1ac0 auth_method 0 2024-03-13T07:33:49.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.711+0000 7fe7a585e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:49.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.711+0000 7fe7a585e640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:49.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.711+0000 7fe7a585e640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:49.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.712+0000 7fe7a505d640 10 monclient(hunting): get_auth_request con 0x7fe7a00b5320 auth_method 0 2024-03-13T07:33:49.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.712+0000 7fe7a505d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:49.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.712+0000 7fe7a505d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:49.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.712+0000 7fe7a505d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:49.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.712+0000 7fe7a605f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:49.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.712+0000 7fe7a605f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:49.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.712+0000 7fe7a605f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:49.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.712+0000 7fe7a585e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:49.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.712+0000 7fe7a585e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:49.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.712+0000 7fe7a585e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:49.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.712+0000 7fe7a505d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:49.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.712+0000 7fe7a505d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:49.719 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.712+0000 7fe7a505d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:49.720 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.714+0000 7fe7a605f640 10 monclient(hunting): handle_auth_done global_id 4114 payload 824 2024-03-13T07:33:49.720 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.714+0000 7fe7a605f640 1 -- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7a00b5320 msgr2=0x7fe7a0141ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:49.720 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.714+0000 7fe7a605f640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7a00b5320 0x7fe7a0141ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:49.720 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.714+0000 7fe7a605f640 1 -- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 msgr2=0x7fe7a013b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:49.720 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.714+0000 7fe7a605f640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 0x7fe7a013b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:49.720 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.714+0000 7fe7a605f640 10 monclient: _finish_hunting 0 2024-03-13T07:33:49.720 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.714+0000 7fe7a605f640 1 monclient: found mon.noname-b 2024-03-13T07:33:49.720 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.714+0000 7fe7a605f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:49.720 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.714+0000 7fe7a605f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:33:49.720 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.714+0000 7fe7a605f640 1 -- 172.21.15.195:0/4156414818 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00a9f10 con 0x7fe7a013f630 2024-03-13T07:33:49.721 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.717+0000 7fe7a605f640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7a013f630 0x7fe7a0142b90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00c390 tx=0x7fe79c00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:49.721 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.717+0000 7fe7967fc640 10 client.?.objecter ms_handle_connect 0x7fe7a013f630 2024-03-13T07:33:49.721 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.717+0000 7fe7967fc640 10 client.?.objecter resend_mon_ops 2024-03-13T07:33:49.724 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7967fc640 1 -- 172.21.15.195:0/4156414818 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c017070 con 0x7fe7a013f630 2024-03-13T07:33:49.724 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:49.724 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7967fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:33:49.724 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7967fc640 10 monclient: dump: 2024-03-13T07:33:49.724 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:49.724 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:49.724 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:49.724 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:49.724 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:49.724 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:49.725 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:49.725 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:49.725 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:49.725 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:49.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7967fc640 10 monclient: _finish_auth 0 2024-03-13T07:33:49.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7967fc640 10 monclient: _check_auth_tickets 2024-03-13T07:33:49.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:49.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7967fc640 1 -- 172.21.15.195:0/4156414818 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79c0052d0 con 0x7fe7a013f630 2024-03-13T07:33:49.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7a7ac2640 5 monclient: authenticate success, global_id 4114 2024-03-13T07:33:49.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7a7ac2640 10 monclient: _renew_subs 2024-03-13T07:33:49.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7a7ac2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:49.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7a7ac2640 1 -- 172.21.15.195:0/4156414818 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a00aa1f0 con 0x7fe7a013f630 2024-03-13T07:33:49.725 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7967fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:49.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7967fc640 1 -- 172.21.15.195:0/4156414818 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c003810 con 0x7fe7a013f630 2024-03-13T07:33:49.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:49.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7967fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:33:49.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7967fc640 10 monclient: dump: 2024-03-13T07:33:49.726 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:49.726 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:49.726 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:49.726 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:49.726 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:49.726 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:49.726 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:49.726 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:49.726 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:49.726 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:49.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe797fff640 4 set_mon_vals no callback set 2024-03-13T07:33:49.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7a7ac2640 10 client.4114.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:33:49.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7a7ac2640 10 monclient: _renew_subs 2024-03-13T07:33:49.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7a7ac2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:49.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7a7ac2640 1 -- 172.21.15.195:0/4156414818 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a01432b0 con 0x7fe7a013f630 2024-03-13T07:33:49.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.721+0000 7fe7a7ac2640 1 librados: init done 2024-03-13T07:33:49.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.722+0000 7fe7a7ac2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:49.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.722+0000 7fe7a7ac2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:49.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.722+0000 7fe7a605f640 1 -- 172.21.15.195:0/4156414818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7a013f630 msgr2=0x7fe7a0142b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-13T07:33:49.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.722+0000 7fe7a7ac2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:49.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.722+0000 7fe7a605f640 1 -- 172.21.15.195:0/4156414818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7a013f630 msgr2=0x7fe7a0142b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-13T07:33:49.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.722+0000 7fe7a605f640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7a013f630 0x7fe7a0142b90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00c390 tx=0x7fe79c00c850 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-13T07:33:49.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.722+0000 7fe7a605f640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7a013f630 0x7fe7a0142b90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00c390 tx=0x7fe79c00c850 comp rx=0 tx=0).stop 2024-03-13T07:33:49.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.722+0000 7fe7a7ac2640 1 -- 172.21.15.195:0/4156414818 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a00af760 con 0x7fe7a013f630 2024-03-13T07:33:49.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.722+0000 7fe7967fc640 10 monclient: ms_handle_reset current mon [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:33:49.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.722+0000 7fe7967fc640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:49.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.722+0000 7fe7967fc640 1 -- 172.21.15.195:0/4156414818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7a013f630 msgr2=0x7fe7a0142b90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-13T07:33:49.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.722+0000 7fe7967fc640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7a013f630 0x7fe7a0142b90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:49.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.722+0000 7fe7967fc640 10 monclient: _add_conns ranks=[0,1,2] 2024-03-13T07:33:49.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.723+0000 7fe7967fc640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 0x7fe784007250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:49.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.723+0000 7fe7967fc640 10 monclient(hunting): picked mon.a con 0x7fe7a00b1ac0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:33:49.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.723+0000 7fe7967fc640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe784007c20 0x7fe78400a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:49.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.723+0000 7fe7967fc640 10 monclient(hunting): picked mon.b con 0x7fe784007c20 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:33:49.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.723+0000 7fe7967fc640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7a00b5320 0x7fe78400cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:49.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.723+0000 7fe7967fc640 10 monclient(hunting): picked mon.c con 0x7fe7a00b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:49.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.723+0000 7fe7967fc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:49.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.723+0000 7fe7967fc640 1 -- 172.21.15.195:0/4156414818 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78400d5e0 con 0x7fe7a00b1ac0 2024-03-13T07:33:49.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.723+0000 7fe7967fc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:49.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.723+0000 7fe7967fc640 1 -- 172.21.15.195:0/4156414818 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78400d760 con 0x7fe784007c20 2024-03-13T07:33:49.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.723+0000 7fe7967fc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:49.729 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.723+0000 7fe7967fc640 1 -- 172.21.15.195:0/4156414818 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78400d8e0 con 0x7fe7a00b5320 2024-03-13T07:33:49.729 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.723+0000 7fe7967fc640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:49.729 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.723+0000 7fe7a585e640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 0x7fe784007250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:49.729 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.723+0000 7fe7a605f640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7a00b5320 0x7fe78400cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:49.729 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.724+0000 7fe7a605f640 10 monclient(hunting): get_auth_request con 0x7fe7a00b5320 auth_method 0 2024-03-13T07:33:49.729 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.724+0000 7fe7a605f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:49.729 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.724+0000 7fe7a605f640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:49.729 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.724+0000 7fe7a605f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-03-13T07:33:49.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.724+0000 7fe7a605f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:49.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.724+0000 7fe7a605f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:49.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.724+0000 7fe7a605f640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-03-13T07:33:49.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.725+0000 7fe7a585e640 1 -- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 msgr2=0x7fe784007250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-13T07:33:49.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.725+0000 7fe7a585e640 1 -- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 msgr2=0x7fe784007250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-13T07:33:49.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.725+0000 7fe7a585e640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 0x7fe784007250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-13T07:33:49.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.725+0000 7fe7a585e640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 0x7fe784007250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T07:33:49.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.727+0000 7fe7a605f640 10 monclient(hunting): handle_auth_done global_id 4114 payload 293 2024-03-13T07:33:49.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.727+0000 7fe7a605f640 1 -- 172.21.15.195:0/4156414818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe784007c20 msgr2=0x7fe78400a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:49.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.727+0000 7fe7a605f640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe784007c20 0x7fe78400a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:49.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.727+0000 7fe7a605f640 1 -- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 msgr2=0x7fe784007250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-13T07:33:49.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.727+0000 7fe7a605f640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 0x7fe784007250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:49.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.727+0000 7fe7a605f640 10 monclient: _finish_hunting 0 2024-03-13T07:33:49.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.727+0000 7fe7a605f640 1 monclient: found mon.c 2024-03-13T07:33:49.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.727+0000 7fe7a605f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:49.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.727+0000 7fe7a605f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:49.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.727+0000 7fe7a605f640 1 -- 172.21.15.195:0/4156414818 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) v3 -- 0x7fe78400dc10 con 0x7fe7a00b5320 2024-03-13T07:33:49.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.727+0000 7fe7a605f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:49.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.727+0000 7fe7a605f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:49.732 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.727+0000 7fe7a605f640 1 -- 172.21.15.195:0/4156414818 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a01432b0 con 0x7fe7a00b5320 2024-03-13T07:33:49.732 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.727+0000 7fe7a605f640 10 monclient: _finish_auth 0 2024-03-13T07:33:49.732 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.727+0000 7fe7a605f640 10 monclient: _check_auth_tickets 2024-03-13T07:33:49.732 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.727+0000 7fe7a605f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:49.732 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.729+0000 7fe7967fc640 10 client.4114.objecter ms_handle_connect 0x7fe7a00b5320 2024-03-13T07:33:49.732 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.729+0000 7fe7967fc640 10 client.4114.objecter resend_mon_ops 2024-03-13T07:33:49.732 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.729+0000 7fe7a605f640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7a00b5320 0x7fe78400cbd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe79c011600 tx=0x7fe79c007e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:49.740 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:49.739+0000 7fd30663e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-13T07:33:49.762 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:49.762+0000 7fb6b5844040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-13T07:33:49.784 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:49.784+0000 7fd30663e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-13T07:33:49.807 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:49.808+0000 7fb6b5844040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-13T07:33:49.827 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.827+0000 7fe7967fc640 1 -- 172.21.15.195:0/4156414818 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c003af0 con 0x7fe7a00b5320 2024-03-13T07:33:49.827 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.827+0000 7fe7967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:49.827 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.827+0000 7fe7967fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:33:49.827 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.827+0000 7fe7967fc640 10 monclient: dump: 2024-03-13T07:33:49.827 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:49.827 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:49.828 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:49.828 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:49.828 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:49.828 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:49.828 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:49.828 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:49.828 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:49.828 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:49.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.827+0000 7fe7967fc640 1 -- 172.21.15.195:0/4156414818 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79c011850 con 0x7fe7a00b5320 2024-03-13T07:33:49.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.827+0000 7fe7967fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:49.828 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.827+0000 7fe7977fe640 4 set_mon_vals no callback set 2024-03-13T07:33:49.831 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.831+0000 7fe7967fc640 1 -- 172.21.15.195:0/4156414818 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe79c01a070 con 0x7fe7a00b5320 2024-03-13T07:33:49.831 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.831+0000 7fe7967fc640 1 -- 172.21.15.195:0/4156414818 <== mon.2 v2:172.21.15.132:3301/0 4 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe79c027d10 con 0x7fe7a00b5320 2024-03-13T07:33:49.831 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.831+0000 7fe7967fc640 10 client.4114.objecter ms_dispatch 0x7fe7a0000b90 osd_map(2..2 src has 1..2) v4 2024-03-13T07:33:49.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.831+0000 7fe7967fc640 3 client.4114.objecter handle_osd_map got epochs [2,2] > 0 2024-03-13T07:33:49.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.831+0000 7fe7967fc640 3 client.4114.objecter handle_osd_map decoding full epoch 2 2024-03-13T07:33:49.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.831+0000 7fe7967fc640 20 client.4114.objecter dump_active .. 0 homeless 2024-03-13T07:33:49.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.831+0000 7fe7967fc640 1 -- 172.21.15.195:0/4156414818 <== mon.2 v2:172.21.15.132:3301/0 5 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe79c021030 con 0x7fe7a00b5320 2024-03-13T07:33:49.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.831+0000 7fe7967fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:49.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.831+0000 7fe7967fc640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:33:49.837 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:49.837+0000 7fd30663e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-13T07:33:49.861 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:49.862+0000 7fb6b5844040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-13T07:33:49.881 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:49.881+0000 7fd30663e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-13T07:33:49.903 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:49.903+0000 7fb6b5844040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-13T07:33:49.904 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.904+0000 7fe7a7ac2640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "0edb216c-bc1e-4865-aafe-218611f3eb09", "id": 0}] 2024-03-13T07:33:49.904 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.904+0000 7fe7a7ac2640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "0edb216c-bc1e-4865-aafe-218611f3eb09", "id": 0}] 2024-03-13T07:33:49.904 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.904+0000 7fe7a7ac2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:49.905 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:49.904+0000 7fe7a7ac2640 1 -- 172.21.15.195:0/4156414818 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0edb216c-bc1e-4865-aafe-218611f3eb09", "id": 0} v 0) v1 -- 0x7fe7a00af760 con 0x7fe7a00b5320 2024-03-13T07:33:49.997 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:49.997+0000 7fd30663e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-13T07:33:50.024 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:50.025+0000 7fb6b5844040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-13T07:33:50.066 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:50.066+0000 7fd30663e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-13T07:33:50.099 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:50.099+0000 7fb6b5844040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-13T07:33:50.114 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:50.114+0000 7fd30663e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-13T07:33:50.149 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:50.149+0000 7fb6b5844040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-13T07:33:50.157 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.157+0000 7fe7967fc640 1 -- 172.21.15.195:0/4156414818 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0edb216c-bc1e-4865-aafe-218611f3eb09", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe7a00af760 con 0x7fe7a00b5320 2024-03-13T07:33:50.158 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.157+0000 7fe7967fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "0edb216c-bc1e-4865-aafe-218611f3eb09", "id": 0}] 2024-03-13T07:33:50.158 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.157+0000 7fe7967fc640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:33:50.158 INFO:teuthology.orchestra.run.smithi195.stdout:0 2024-03-13T07:33:50.159 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.158+0000 7fe7a7ac2640 10 librados: watch_flush enter 2024-03-13T07:33:50.159 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.158+0000 7fe7a7ac2640 10 librados: watch_flush exit 2024-03-13T07:33:50.159 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.158+0000 7fe7a7ac2640 20 client.4114.objecter shutdown clearing up homeless session... 2024-03-13T07:33:50.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.158+0000 7fe7a7ac2640 10 client.4114.objecter successfully canceled tick 2024-03-13T07:33:50.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.158+0000 7fe7a7ac2640 10 monclient: shutdown 2024-03-13T07:33:50.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.158+0000 7fe7a7ac2640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:50.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.158+0000 7fe7a7ac2640 1 -- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7a00b5320 msgr2=0x7fe78400cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:50.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.158+0000 7fe7a7ac2640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7a00b5320 0x7fe78400cbd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe79c011600 tx=0x7fe79c007e40 comp rx=0 tx=0).stop 2024-03-13T07:33:50.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.158+0000 7fe7a7ac2640 1 -- 172.21.15.195:0/4156414818 shutdown_connections 2024-03-13T07:33:50.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.158+0000 7fe7a7ac2640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7a00b5320 0x7fe78400cbd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:50.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.158+0000 7fe7a7ac2640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe784007c20 0x7fe78400a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:50.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.158+0000 7fe7a7ac2640 1 --2- 172.21.15.195:0/4156414818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a00b1ac0 0x7fe784007250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:50.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.158+0000 7fe7a7ac2640 1 -- 172.21.15.195:0/4156414818 >> 172.21.15.195:0/4156414818 conn(0x7fe7a006d2b0 msgr2=0x7fe7a013fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:50.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.158+0000 7fe7a7ac2640 1 -- 172.21.15.195:0/4156414818 shutdown_connections 2024-03-13T07:33:50.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.158+0000 7fe7a7ac2640 1 -- 172.21.15.195:0/4156414818 wait complete. 2024-03-13T07:33:50.160 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.159+0000 7fe7a7ac2640 1 librados: shutdown 2024-03-13T07:33:50.169 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 6be0ae6b-951e-4279-ba0a-9b05b6e6e85e 1 2024-03-13T07:33:50.169 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:50.168+0000 7fd30663e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-13T07:33:50.207 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:50.207+0000 7fb6b5844040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-13T07:33:50.220 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:50.219+0000 7fd30663e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-13T07:33:50.256 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:50.256+0000 7fb6b5844040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-13T07:33:50.270 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.269+0000 7f1c6099a640 10 monclient: get_monmap_and_config 2024-03-13T07:33:50.270 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.269+0000 7f1c6099a640 10 monclient: build_initial_monmap 2024-03-13T07:33:50.270 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.270+0000 7f1c6099a640 10 monclient: monmap: 2024-03-13T07:33:50.270 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 0 2024-03-13T07:33:50.270 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:50.270 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:50.271058+0000 2024-03-13T07:33:50.270 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:50.271058+0000 2024-03-13T07:33:50.270 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:50.270 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:50.270 INFO:teuthology.orchestra.run.smithi195.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:33:50.271 INFO:teuthology.orchestra.run.smithi195.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:33:50.271 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:33:50.271 INFO:teuthology.orchestra.run.smithi195.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:33:50.271 INFO:teuthology.orchestra.run.smithi195.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:33:50.271 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:50.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.270+0000 7f1c6099a640 1 Processor -- start 2024-03-13T07:33:50.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.270+0000 7f1c6099a640 1 -- start start 2024-03-13T07:33:50.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.270+0000 7f1c6099a640 10 monclient: init 2024-03-13T07:33:50.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.271+0000 7f1c6099a640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:50.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.271+0000 7f1c6099a640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2024-03-13T07:33:50.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.271+0000 7f1c6099a640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0b5320 0x7f1c5c0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:50.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.271+0000 7f1c6099a640 10 monclient(hunting): picked mon.noname-b con 0x7f1c5c0b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:50.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.271+0000 7f1c6099a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1c5c0b1ac0 addr v1:172.21.15.195:6789/0 2024-03-13T07:33:50.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.271+0000 7f1c6099a640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1c5c13fd40 0x7f1c5c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:50.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.271+0000 7f1c6099a640 10 monclient(hunting): picked mon.noname-a con 0x7f1c5c13fd40 addr v2:172.21.15.132:3300/0 2024-03-13T07:33:50.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.271+0000 7f1c6099a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5c0b1150 con 0x7f1c5c0b1ac0 2024-03-13T07:33:50.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.271+0000 7f1c6099a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:50.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.271+0000 7f1c6099a640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c0b5700 con 0x7f1c5c13fd40 2024-03-13T07:33:50.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.271+0000 7f1c6099a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:50.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.271+0000 7f1c6099a640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0b13c0 con 0x7f1c5c0b5320 2024-03-13T07:33:50.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.271+0000 7f1c6099a640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:50.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.271+0000 7f1c6099a640 10 monclient(hunting): authenticate will time out at 1244.609839s 2024-03-13T07:33:50.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.272+0000 7f1c59d9b640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f1c5c0b1ac0 0x7f1c5c13f630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.195:55996/0 (socket says 172.21.15.195:55996) 2024-03-13T07:33:50.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.272+0000 7f1c59d9b640 1 -- 172.21.15.195:0/2384506999 learned_addr learned my addr 172.21.15.195:0/2384506999 (peer_addr_for_me v1:172.21.15.195:0/0) 2024-03-13T07:33:50.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.272+0000 7f1c5a59c640 1 --2- 172.21.15.195:0/2384506999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0b5320 0x7f1c5c0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:50.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.272+0000 7f1c5ad9d640 1 --2- 172.21.15.195:0/2384506999 >> v2:172.21.15.132:3300/0 conn(0x7f1c5c13fd40 0x7f1c5c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:50.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.272+0000 7f1c5ad9d640 10 monclient(hunting): get_auth_request con 0x7f1c5c13fd40 auth_method 0 2024-03-13T07:33:50.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.272+0000 7f1c5ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:50.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.272+0000 7f1c5ad9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:50.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.272+0000 7f1c5ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:50.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.272+0000 7f1c5a59c640 10 monclient(hunting): get_auth_request con 0x7f1c5c0b5320 auth_method 0 2024-03-13T07:33:50.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.272+0000 7f1c5a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:50.274 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.272+0000 7f1c5a59c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:50.275 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.272+0000 7f1c5a59c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:50.276 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.273+0000 7f1c5a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:50.276 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.273+0000 7f1c5a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:50.276 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.273+0000 7f1c5a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:50.276 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.273+0000 7f1c5ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:50.276 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.273+0000 7f1c5ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:50.276 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.273+0000 7f1c5ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:50.276 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.275+0000 7f1c5959a640 1 -- 172.21.15.195:0/2384506999 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973221334 0 0) 0x7f1c5c0b1150 con 0x7f1c5c0b1ac0 2024-03-13T07:33:50.276 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.275+0000 7f1c5959a640 10 monclient(hunting): _negotiate 2024-03-13T07:33:50.276 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.275+0000 7f1c5959a640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:50.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.275+0000 7f1c5959a640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:50.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.275+0000 7f1c5959a640 10 monclient(hunting): my global_id is 4117 2024-03-13T07:33:50.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.275+0000 7f1c5959a640 1 -- 172.21.15.195:0/2384506999 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c44003520 con 0x7f1c5c0b1ac0 2024-03-13T07:33:50.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.275+0000 7f1c5ad9d640 10 monclient(hunting): handle_auth_done global_id 4113 payload 274 2024-03-13T07:33:50.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.275+0000 7f1c5ad9d640 1 -- 172.21.15.195:0/2384506999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0b5320 msgr2=0x7f1c5c0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:33:50.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.275+0000 7f1c5ad9d640 1 --2- 172.21.15.195:0/2384506999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0b5320 0x7f1c5c0b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:50.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.275+0000 7f1c5ad9d640 1 -- 172.21.15.195:0/2384506999 >> v1:172.21.15.195:6789/0 conn(0x7f1c5c0b1ac0 legacy=0x7f1c5c13f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:50.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.275+0000 7f1c5ad9d640 10 monclient: _finish_hunting 0 2024-03-13T07:33:50.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.275+0000 7f1c5ad9d640 1 monclient: found mon.noname-a 2024-03-13T07:33:50.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.275+0000 7f1c5ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:50.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.275+0000 7f1c5ad9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:33:50.277 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.275+0000 7f1c5ad9d640 1 -- 172.21.15.195:0/2384506999 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c0b16a0 con 0x7f1c5c13fd40 2024-03-13T07:33:50.278 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.278+0000 7f1c5ad9d640 1 --2- 172.21.15.195:0/2384506999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c5c13fd40 0x7f1c5c0b1f00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0097a0 tx=0x7f1c4c02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7c10ebc4861573 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:50.282 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c5959a640 1 -- 172.21.15.195:0/2384506999 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c038070 con 0x7f1c5c13fd40 2024-03-13T07:33:50.282 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c5959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:50.282 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c5959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:33:50.282 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c5959a640 10 monclient: dump: 2024-03-13T07:33:50.282 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:50.282 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:50.282 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:50.282 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:50.283 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:50.283 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:50.283 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:50.283 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:50.283 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:50.283 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:50.283 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c5959a640 10 monclient: _finish_auth 0 2024-03-13T07:33:50.283 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c5959a640 10 monclient: _check_auth_tickets 2024-03-13T07:33:50.283 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c5959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:50.283 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c5959a640 1 -- 172.21.15.195:0/2384506999 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4c031700 con 0x7f1c5c13fd40 2024-03-13T07:33:50.283 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c5959a640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:50.284 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c5959a640 1 -- 172.21.15.195:0/2384506999 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c031ad0 con 0x7f1c5c13fd40 2024-03-13T07:33:50.284 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c5959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:50.284 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c5959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:33:50.284 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c5959a640 10 monclient: dump: 2024-03-13T07:33:50.284 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:50.284 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:50.284 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:50.284 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:50.284 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:50.284 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:50.284 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:50.284 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:50.284 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:50.284 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:50.284 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c6099a640 5 monclient: authenticate success, global_id 4113 2024-03-13T07:33:50.284 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c6099a640 10 monclient: get_monmap_and_config success 2024-03-13T07:33:50.285 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c6099a640 4 set_mon_vals no callback set 2024-03-13T07:33:50.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c6099a640 10 monclient: shutdown 2024-03-13T07:33:50.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.282+0000 7f1c6099a640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:50.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.283+0000 7f1c6099a640 1 -- 172.21.15.195:0/2384506999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c5c13fd40 msgr2=0x7f1c5c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:50.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.283+0000 7f1c6099a640 1 --2- 172.21.15.195:0/2384506999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c5c13fd40 0x7f1c5c0b1f00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0097a0 tx=0x7f1c4c02b7a0 comp rx=0 tx=0).stop 2024-03-13T07:33:50.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.283+0000 7f1c6099a640 1 -- 172.21.15.195:0/2384506999 shutdown_connections 2024-03-13T07:33:50.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.283+0000 7f1c6099a640 1 --2- 172.21.15.195:0/2384506999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c5c13fd40 0x7f1c5c0b1f00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:50.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.283+0000 7f1c6099a640 1 --2- 172.21.15.195:0/2384506999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0b5320 0x7f1c5c0b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:50.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.283+0000 7f1c6099a640 1 -- 172.21.15.195:0/2384506999 >> 172.21.15.195:0/2384506999 conn(0x7f1c5c06d2b0 msgr2=0x7f1c5c06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:50.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.283+0000 7f1c6099a640 1 -- 172.21.15.195:0/2384506999 shutdown_connections 2024-03-13T07:33:50.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.283+0000 7f1c6099a640 1 -- 172.21.15.195:0/2384506999 wait complete. 2024-03-13T07:33:50.289 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.283+0000 7f1c6099a640 10 monclient: build_initial_monmap 2024-03-13T07:33:50.289 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.283+0000 7f1c6099a640 10 monclient: monmap: 2024-03-13T07:33:50.289 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 0 2024-03-13T07:33:50.289 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:33:50.289 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 0.000000 2024-03-13T07:33:50.289 INFO:teuthology.orchestra.run.smithi195.stderr:created 0.000000 2024-03-13T07:33:50.289 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:50.289 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:50.289 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:33:50.289 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:33:50.289 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:33:50.289 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:50.289 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 1 librados: starting msgr at 2024-03-13T07:33:50.289 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 1 librados: starting objecter 2024-03-13T07:33:50.289 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 1 Processor -- start 2024-03-13T07:33:50.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 1 -- start start 2024-03-13T07:33:50.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 1 librados: setting wanted keys 2024-03-13T07:33:50.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 1 librados: calling monclient init 2024-03-13T07:33:50.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 10 monclient: init 2024-03-13T07:33:50.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:50.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 10 monclient: _add_conns ranks=[2,0,1] 2024-03-13T07:33:50.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0b1ac0 0x7f1c5c142610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:50.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 10 monclient(hunting): picked mon.noname-c con 0x7f1c5c0b1ac0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:50.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c5c0b5320 0x7f1c5c0aa980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:50.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 10 monclient(hunting): picked mon.noname-a con 0x7f1c5c0b5320 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:33:50.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c5c13fd40 0x7f1c5c0ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:50.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 10 monclient(hunting): picked mon.noname-b con 0x7f1c5c13fd40 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:33:50.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:50.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0b4800 con 0x7f1c5c0b5320 2024-03-13T07:33:50.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:50.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0afae0 con 0x7f1c5c13fd40 2024-03-13T07:33:50.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:50.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0b4980 con 0x7f1c5c0b1ac0 2024-03-13T07:33:50.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:50.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.284+0000 7f1c6099a640 10 monclient(hunting): authenticate will time out at 1244.623344s 2024-03-13T07:33:50.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.285+0000 7f1c59d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c5c0b5320 0x7f1c5c0aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:50.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.285+0000 7f1c5a59c640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0b1ac0 0x7f1c5c142610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:50.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.285+0000 7f1c5ad9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c5c13fd40 0x7f1c5c0ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:50.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.285+0000 7f1c59d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c5c0b5320 0x7f1c5c0aa980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.195:52584/0 (socket says 172.21.15.195:52584) 2024-03-13T07:33:50.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.285+0000 7f1c5a59c640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0b1ac0 0x7f1c5c142610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.195:48736/0 (socket says 172.21.15.195:48736) 2024-03-13T07:33:50.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.285+0000 7f1c59d9b640 1 -- 172.21.15.195:0/3746276912 learned_addr learned my addr 172.21.15.195:0/3746276912 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-13T07:33:50.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.285+0000 7f1c59d9b640 10 monclient(hunting): get_auth_request con 0x7f1c5c0b5320 auth_method 0 2024-03-13T07:33:50.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.285+0000 7f1c59d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:50.292 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.285+0000 7f1c59d9b640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:50.292 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.285+0000 7f1c59d9b640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:50.292 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.285+0000 7f1c5a59c640 10 monclient(hunting): get_auth_request con 0x7f1c5c0b1ac0 auth_method 0 2024-03-13T07:33:50.292 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.285+0000 7f1c5a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:50.292 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.285+0000 7f1c5a59c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:50.292 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.285+0000 7f1c5a59c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:50.292 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.286+0000 7f1c5ad9d640 10 monclient(hunting): get_auth_request con 0x7f1c5c13fd40 auth_method 0 2024-03-13T07:33:50.292 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.286+0000 7f1c5ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:50.292 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.286+0000 7f1c5ad9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:50.292 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.286+0000 7f1c5ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:50.292 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.286+0000 7f1c5a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:50.292 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.286+0000 7f1c5a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:50.292 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.286+0000 7f1c5a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:50.292 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.286+0000 7f1c59d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:50.292 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.286+0000 7f1c59d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:50.293 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.286+0000 7f1c59d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:50.293 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.286+0000 7f1c5ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:50.293 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.286+0000 7f1c5ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:50.293 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.286+0000 7f1c5ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:50.293 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.289+0000 7f1c5ad9d640 10 monclient(hunting): handle_auth_done global_id 4120 payload 824 2024-03-13T07:33:50.293 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.289+0000 7f1c5ad9d640 1 -- 172.21.15.195:0/3746276912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0b1ac0 msgr2=0x7f1c5c142610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:50.294 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.289+0000 7f1c5ad9d640 1 --2- 172.21.15.195:0/3746276912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0b1ac0 0x7f1c5c142610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:50.294 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.289+0000 7f1c5ad9d640 1 -- 172.21.15.195:0/3746276912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c5c0b5320 msgr2=0x7f1c5c0aa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:50.294 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.289+0000 7f1c5ad9d640 1 --2- 172.21.15.195:0/3746276912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c5c0b5320 0x7f1c5c0aa980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:50.294 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.289+0000 7f1c5ad9d640 10 monclient: _finish_hunting 0 2024-03-13T07:33:50.294 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.289+0000 7f1c5ad9d640 1 monclient: found mon.noname-b 2024-03-13T07:33:50.294 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.289+0000 7f1c5ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:50.294 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.289+0000 7f1c5ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:33:50.294 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.289+0000 7f1c5ad9d640 1 -- 172.21.15.195:0/3746276912 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c0acd00 con 0x7f1c5c13fd40 2024-03-13T07:33:50.294 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.291+0000 7f1c5ad9d640 1 --2- 172.21.15.195:0/3746276912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c5c13fd40 0x7f1c5c0ab620 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c069d80 tx=0x7f1c4c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:50.294 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.291+0000 7f1c4affd640 10 client.?.objecter ms_handle_connect 0x7f1c5c13fd40 2024-03-13T07:33:50.294 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.291+0000 7f1c4affd640 10 client.?.objecter resend_mon_ops 2024-03-13T07:33:50.297 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c4affd640 1 -- 172.21.15.195:0/3746276912 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c038040 con 0x7f1c5c13fd40 2024-03-13T07:33:50.297 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c4affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:50.297 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c4affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:33:50.297 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c4affd640 10 monclient: dump: 2024-03-13T07:33:50.297 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:50.297 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:50.297 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:50.297 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:50.297 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:50.297 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:50.297 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:50.298 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:50.298 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:50.298 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:50.298 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c4affd640 10 monclient: _finish_auth 0 2024-03-13T07:33:50.298 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c4affd640 10 monclient: _check_auth_tickets 2024-03-13T07:33:50.298 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c4affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:50.298 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c4affd640 1 -- 172.21.15.195:0/3746276912 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4c03c610 con 0x7f1c5c13fd40 2024-03-13T07:33:50.298 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c6099a640 5 monclient: authenticate success, global_id 4120 2024-03-13T07:33:50.298 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c6099a640 10 monclient: _renew_subs 2024-03-13T07:33:50.298 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c6099a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:50.298 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c6099a640 1 -- 172.21.15.195:0/3746276912 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c0acfe0 con 0x7f1c5c13fd40 2024-03-13T07:33:50.298 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c4affd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:50.298 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c4affd640 1 -- 172.21.15.195:0/3746276912 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c03ca00 con 0x7f1c5c13fd40 2024-03-13T07:33:50.298 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c4affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:50.299 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c4affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:33:50.299 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c4affd640 10 monclient: dump: 2024-03-13T07:33:50.299 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:50.299 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:50.299 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:50.299 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:50.299 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:50.299 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:50.299 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:50.299 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:50.299 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:50.299 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:50.299 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c58d99640 4 set_mon_vals no callback set 2024-03-13T07:33:50.299 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c6099a640 10 client.4120.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:33:50.299 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c6099a640 10 monclient: _renew_subs 2024-03-13T07:33:50.299 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c6099a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:50.300 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c6099a640 1 -- 172.21.15.195:0/3746276912 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c0ad650 con 0x7f1c5c13fd40 2024-03-13T07:33:50.300 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.296+0000 7f1c6099a640 1 librados: init done 2024-03-13T07:33:50.300 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.297+0000 7f1c6099a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:50.300 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.297+0000 7f1c6099a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:50.300 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.297+0000 7f1c6099a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:50.300 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.297+0000 7f1c6099a640 1 -- 172.21.15.195:0/3746276912 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c04f000 con 0x7f1c5c13fd40 2024-03-13T07:33:50.301 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.300+0000 7f1c4affd640 1 -- 172.21.15.195:0/3746276912 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1c4c045020 con 0x7f1c5c13fd40 2024-03-13T07:33:50.301 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.300+0000 7f1c4affd640 1 -- 172.21.15.195:0/3746276912 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f1c4c051bd0 con 0x7f1c5c13fd40 2024-03-13T07:33:50.301 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.300+0000 7f1c4affd640 10 client.4120.objecter ms_dispatch 0x7f1c5c000b90 osd_map(3..3 src has 1..3) v4 2024-03-13T07:33:50.301 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.300+0000 7f1c4affd640 3 client.4120.objecter handle_osd_map got epochs [3,3] > 0 2024-03-13T07:33:50.301 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.300+0000 7f1c4affd640 3 client.4120.objecter handle_osd_map decoding full epoch 3 2024-03-13T07:33:50.301 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.300+0000 7f1c4affd640 20 client.4120.objecter dump_active .. 0 homeless 2024-03-13T07:33:50.495 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.494+0000 7f1c4affd640 1 -- 172.21.15.195:0/3746276912 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1c4c030070 con 0x7f1c5c13fd40 2024-03-13T07:33:50.495 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.494+0000 7f1c4affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:50.495 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.494+0000 7f1c4affd640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:33:50.568 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.568+0000 7f1c6099a640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "6be0ae6b-951e-4279-ba0a-9b05b6e6e85e", "id": 1}] 2024-03-13T07:33:50.568 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.568+0000 7f1c6099a640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "6be0ae6b-951e-4279-ba0a-9b05b6e6e85e", "id": 1}] 2024-03-13T07:33:50.568 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.568+0000 7f1c6099a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:50.568 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.568+0000 7f1c6099a640 1 -- 172.21.15.195:0/3746276912 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6be0ae6b-951e-4279-ba0a-9b05b6e6e85e", "id": 1} v 0) v1 -- 0x7f1c5c067e10 con 0x7f1c5c13fd40 2024-03-13T07:33:50.640 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:50.640+0000 7fd30663e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-13T07:33:50.640 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:50.640+0000 7fb6b5844040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-13T07:33:50.715 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.715+0000 7f1c4affd640 1 -- 172.21.15.195:0/3746276912 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6be0ae6b-951e-4279-ba0a-9b05b6e6e85e", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1c4c051e20 con 0x7f1c5c13fd40 2024-03-13T07:33:50.715 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.715+0000 7f1c4affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "6be0ae6b-951e-4279-ba0a-9b05b6e6e85e", "id": 1}] 2024-03-13T07:33:50.715 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.715+0000 7f1c4affd640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:33:50.715 INFO:teuthology.orchestra.run.smithi195.stdout:1 2024-03-13T07:33:50.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.715+0000 7f1c6099a640 10 librados: watch_flush enter 2024-03-13T07:33:50.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.715+0000 7f1c6099a640 10 librados: watch_flush exit 2024-03-13T07:33:50.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.716+0000 7f1c6099a640 20 client.4120.objecter shutdown clearing up homeless session... 2024-03-13T07:33:50.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.716+0000 7f1c6099a640 10 client.4120.objecter successfully canceled tick 2024-03-13T07:33:50.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.716+0000 7f1c6099a640 10 monclient: shutdown 2024-03-13T07:33:50.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.716+0000 7f1c6099a640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:50.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.716+0000 7f1c6099a640 1 -- 172.21.15.195:0/3746276912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c5c13fd40 msgr2=0x7f1c5c0ab620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:50.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.716+0000 7f1c6099a640 1 --2- 172.21.15.195:0/3746276912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c5c13fd40 0x7f1c5c0ab620 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c069d80 tx=0x7f1c4c0046c0 comp rx=0 tx=0).stop 2024-03-13T07:33:50.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.716+0000 7f1c6099a640 1 -- 172.21.15.195:0/3746276912 shutdown_connections 2024-03-13T07:33:50.717 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.716+0000 7f1c6099a640 1 --2- 172.21.15.195:0/3746276912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c5c13fd40 0x7f1c5c0ab620 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:50.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.716+0000 7f1c6099a640 1 --2- 172.21.15.195:0/3746276912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c5c0b5320 0x7f1c5c0aa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:50.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.716+0000 7f1c6099a640 1 --2- 172.21.15.195:0/3746276912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0b1ac0 0x7f1c5c142610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:50.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.716+0000 7f1c6099a640 1 -- 172.21.15.195:0/3746276912 >> 172.21.15.195:0/3746276912 conn(0x7f1c5c06d2b0 msgr2=0x7f1c5c0b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:50.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.716+0000 7f1c6099a640 1 -- 172.21.15.195:0/3746276912 shutdown_connections 2024-03-13T07:33:50.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.716+0000 7f1c6099a640 1 -- 172.21.15.195:0/3746276912 wait complete. 2024-03-13T07:33:50.718 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.716+0000 7f1c6099a640 1 librados: shutdown 2024-03-13T07:33:50.727 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 18008b7f-cfe7-4450-a3cf-aab63ebbe99d 2 2024-03-13T07:33:50.780 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:50.780+0000 7fb6b5844040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-13T07:33:50.786 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:50.784+0000 7fd30663e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-13T07:33:50.826 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.826+0000 7fa9ea5c5640 10 monclient: get_monmap_and_config 2024-03-13T07:33:50.826 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.826+0000 7fa9ea5c5640 10 monclient: build_initial_monmap 2024-03-13T07:33:50.826 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.826+0000 7fa9ea5c5640 10 monclient: monmap: 2024-03-13T07:33:50.827 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 0 2024-03-13T07:33:50.827 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:50.827 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:50.827558+0000 2024-03-13T07:33:50.827 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:50.827558+0000 2024-03-13T07:33:50.827 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:50.827 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:50.827 INFO:teuthology.orchestra.run.smithi195.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:33:50.827 INFO:teuthology.orchestra.run.smithi195.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:33:50.827 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:33:50.827 INFO:teuthology.orchestra.run.smithi195.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:33:50.827 INFO:teuthology.orchestra.run.smithi195.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:33:50.827 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:50.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.827+0000 7fa9ea5c5640 1 Processor -- start 2024-03-13T07:33:50.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.827+0000 7fa9ea5c5640 1 -- start start 2024-03-13T07:33:50.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.827+0000 7fa9ea5c5640 10 monclient: init 2024-03-13T07:33:50.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.827+0000 7fa9ea5c5640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:50.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.827+0000 7fa9ea5c5640 10 monclient: _add_conns ranks=[4,1,0,3,2] 2024-03-13T07:33:50.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.827+0000 7fa9ea5c5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa9e40b5320 0x7fa9e40b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:50.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.827+0000 7fa9ea5c5640 10 monclient(hunting): picked mon.noname-c con 0x7fa9e40b5320 addr v2:172.21.15.195:3300/0 2024-03-13T07:33:50.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.827+0000 7fa9ea5c5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa9e40b1ac0 addr v1:172.21.15.195:6789/0 2024-03-13T07:33:50.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.827+0000 7fa9ea5c5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa9e413fd40 addr v1:172.21.15.132:6789/0 2024-03-13T07:33:50.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.827+0000 7fa9ea5c5640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e40b1150 con 0x7fa9e413fd40 2024-03-13T07:33:50.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.827+0000 7fa9ea5c5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e40b13c0 con 0x7fa9e40b1ac0 2024-03-13T07:33:50.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.827+0000 7fa9ea5c5640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:50.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.827+0000 7fa9ea5c5640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e40b16a0 con 0x7fa9e40b5320 2024-03-13T07:33:50.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.827+0000 7fa9ea5c5640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:50.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.827+0000 7fa9ea5c5640 10 monclient(hunting): authenticate will time out at 1245.166232s 2024-03-13T07:33:50.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.828+0000 7fa9e3fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa9e40b5320 0x7fa9e40b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:50.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.828+0000 7fa9e8b62640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fa9e413fd40 0x7fa9e414cef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.195:45070/0 (socket says 172.21.15.195:45070) 2024-03-13T07:33:50.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.828+0000 7fa9e8b62640 1 -- 172.21.15.195:0/1403235340 learned_addr learned my addr 172.21.15.195:0/1403235340 (peer_addr_for_me v1:172.21.15.195:0/0) 2024-03-13T07:33:50.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.829+0000 7fa9e3fff640 10 monclient(hunting): get_auth_request con 0x7fa9e40b5320 auth_method 0 2024-03-13T07:33:50.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.829+0000 7fa9e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:50.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.829+0000 7fa9e3fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:50.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.829+0000 7fa9e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:50.831 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.829+0000 7fa9e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:50.831 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.829+0000 7fa9e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:50.831 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.829+0000 7fa9e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:50.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.831+0000 7fa9e2ffd640 1 -- 172.21.15.195:0/1403235340 <== mon.0 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263191630 0 0) 0x7fa9e40b1150 con 0x7fa9e413fd40 2024-03-13T07:33:50.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.831+0000 7fa9e2ffd640 10 monclient(hunting): _negotiate 2024-03-13T07:33:50.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.831+0000 7fa9e2ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:50.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.831+0000 7fa9e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:50.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.831+0000 7fa9e2ffd640 10 monclient(hunting): my global_id is 4119 2024-03-13T07:33:50.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.831+0000 7fa9e2ffd640 1 -- 172.21.15.195:0/1403235340 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9cc003520 con 0x7fa9e413fd40 2024-03-13T07:33:50.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.831+0000 7fa9e3fff640 10 monclient(hunting): handle_auth_done global_id 4123 payload 274 2024-03-13T07:33:50.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.831+0000 7fa9e3fff640 1 -- 172.21.15.195:0/1403235340 >> v1:172.21.15.195:6789/0 conn(0x7fa9e40b1ac0 legacy=0x7fa9e413f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:50.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.831+0000 7fa9e3fff640 1 -- 172.21.15.195:0/1403235340 >> v1:172.21.15.132:6789/0 conn(0x7fa9e413fd40 legacy=0x7fa9e414cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:50.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.831+0000 7fa9e3fff640 10 monclient: _finish_hunting 0 2024-03-13T07:33:50.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.831+0000 7fa9e3fff640 1 monclient: found mon.noname-c 2024-03-13T07:33:50.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.831+0000 7fa9e3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:50.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.831+0000 7fa9e3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T07:33:50.832 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.831+0000 7fa9e3fff640 1 -- 172.21.15.195:0/1403235340 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e40afa20 con 0x7fa9e40b5320 2024-03-13T07:33:50.836 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.836+0000 7fa9e3fff640 1 --2- 172.21.15.195:0/1403235340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e40b5320 0x7fa9e40b0220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40097a0 tx=0x7fa9d402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb1ce352feb08fd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:50.845 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9e2ffd640 1 -- 172.21.15.195:0/1403235340 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40315a0 con 0x7fa9e40b5320 2024-03-13T07:33:50.845 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9e2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:50.845 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9e2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:33:50.845 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9e2ffd640 10 monclient: dump: 2024-03-13T07:33:50.846 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:50.846 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:50.846 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:50.846 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:50.846 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:50.846 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:50.846 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:50.846 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:50.846 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:50.847 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:50.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9e2ffd640 10 monclient: _finish_auth 0 2024-03-13T07:33:50.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9e2ffd640 10 monclient: _check_auth_tickets 2024-03-13T07:33:50.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9e2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:50.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9e2ffd640 1 -- 172.21.15.195:0/1403235340 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4037070 con 0x7fa9e40b5320 2024-03-13T07:33:50.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9e2ffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:50.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9e2ffd640 1 -- 172.21.15.195:0/1403235340 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4031960 con 0x7fa9e40b5320 2024-03-13T07:33:50.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9ea5c5640 5 monclient: authenticate success, global_id 4123 2024-03-13T07:33:50.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9ea5c5640 10 monclient: get_monmap_and_config success 2024-03-13T07:33:50.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9ea5c5640 4 set_mon_vals no callback set 2024-03-13T07:33:50.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9e2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:50.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9e2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:33:50.847 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9e2ffd640 10 monclient: dump: 2024-03-13T07:33:50.847 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:50.847 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:50.848 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:50.848 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:50.848 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:50.848 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:50.848 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:50.848 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:50.848 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:50.848 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:50.848 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9ea5c5640 10 monclient: shutdown 2024-03-13T07:33:50.848 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9ea5c5640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:50.848 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9ea5c5640 1 -- 172.21.15.195:0/1403235340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e40b5320 msgr2=0x7fa9e40b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:50.848 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9ea5c5640 1 --2- 172.21.15.195:0/1403235340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e40b5320 0x7fa9e40b0220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40097a0 tx=0x7fa9d402b840 comp rx=0 tx=0).stop 2024-03-13T07:33:50.848 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9ea5c5640 1 -- 172.21.15.195:0/1403235340 shutdown_connections 2024-03-13T07:33:50.848 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9ea5c5640 1 --2- 172.21.15.195:0/1403235340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e40b5320 0x7fa9e40b0220 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:50.848 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.840+0000 7fa9ea5c5640 1 -- 172.21.15.195:0/1403235340 >> 172.21.15.195:0/1403235340 conn(0x7fa9e406d2b0 msgr2=0x7fa9e406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:50.849 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.841+0000 7fa9ea5c5640 1 -- 172.21.15.195:0/1403235340 shutdown_connections 2024-03-13T07:33:50.849 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.841+0000 7fa9ea5c5640 1 -- 172.21.15.195:0/1403235340 wait complete. 2024-03-13T07:33:50.849 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.841+0000 7fa9ea5c5640 10 monclient: build_initial_monmap 2024-03-13T07:33:50.849 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.841+0000 7fa9ea5c5640 10 monclient: monmap: 2024-03-13T07:33:50.849 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 0 2024-03-13T07:33:50.849 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:33:50.849 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 0.000000 2024-03-13T07:33:50.849 INFO:teuthology.orchestra.run.smithi195.stderr:created 0.000000 2024-03-13T07:33:50.849 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:50.849 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:50.849 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:33:50.849 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:33:50.849 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:33:50.849 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:50.849 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.842+0000 7fa9ea5c5640 1 librados: starting msgr at 2024-03-13T07:33:50.850 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.842+0000 7fa9ea5c5640 1 librados: starting objecter 2024-03-13T07:33:50.850 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.842+0000 7fa9ea5c5640 1 Processor -- start 2024-03-13T07:33:50.850 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.842+0000 7fa9ea5c5640 1 -- start start 2024-03-13T07:33:50.850 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.842+0000 7fa9ea5c5640 1 librados: setting wanted keys 2024-03-13T07:33:50.850 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.842+0000 7fa9ea5c5640 1 librados: calling monclient init 2024-03-13T07:33:50.850 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.842+0000 7fa9ea5c5640 10 monclient: init 2024-03-13T07:33:50.850 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9ea5c5640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:50.850 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9ea5c5640 10 monclient: _add_conns ranks=[0,1,2] 2024-03-13T07:33:50.850 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9ea5c5640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9e40b1ac0 0x7fa9e41428a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:50.850 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9ea5c5640 10 monclient(hunting): picked mon.noname-a con 0x7fa9e40b1ac0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:33:50.850 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9ea5c5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e40b5320 0x7fa9e4145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:50.850 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9ea5c5640 10 monclient(hunting): picked mon.noname-b con 0x7fa9e40b5320 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:33:50.850 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9ea5c5640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9e413fd40 0x7fa9e4138200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:50.850 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9ea5c5640 10 monclient(hunting): picked mon.noname-c con 0x7fa9e413fd40 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:50.850 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9ea5c5640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:50.851 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9ea5c5640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e40b4980 con 0x7fa9e40b1ac0 2024-03-13T07:33:50.851 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9ea5c5640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:50.851 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9ea5c5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e40b4800 con 0x7fa9e40b5320 2024-03-13T07:33:50.851 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9ea5c5640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:50.851 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9ea5c5640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e40b1f30 con 0x7fa9e413fd40 2024-03-13T07:33:50.851 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9ea5c5640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:50.851 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9ea5c5640 10 monclient(hunting): authenticate will time out at 1245.181477s 2024-03-13T07:33:50.851 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9e37fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e40b5320 0x7fa9e4145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:50.851 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9e3fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9e40b1ac0 0x7fa9e41428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:50.851 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9e8b62640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9e413fd40 0x7fa9e4138200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:50.851 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9e37fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e40b5320 0x7fa9e4145f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:49840/0 (socket says 172.21.15.195:49840) 2024-03-13T07:33:50.851 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9e37fe640 1 -- 172.21.15.195:0/3618394749 learned_addr learned my addr 172.21.15.195:0/3618394749 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-13T07:33:50.851 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9e37fe640 10 monclient(hunting): get_auth_request con 0x7fa9e40b5320 auth_method 0 2024-03-13T07:33:50.852 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:50.852 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9e37fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:50.852 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.843+0000 7fa9e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:50.852 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e3fff640 10 monclient(hunting): get_auth_request con 0x7fa9e40b1ac0 auth_method 0 2024-03-13T07:33:50.852 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:50.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e3fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:50.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:50.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e8b62640 10 monclient(hunting): get_auth_request con 0x7fa9e413fd40 auth_method 0 2024-03-13T07:33:50.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e8b62640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:50.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e8b62640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:50.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e8b62640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:50.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:50.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:50.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:50.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e8b62640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:50.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e8b62640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:50.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e8b62640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:50.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:50.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:50.853 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.844+0000 7fa9e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:50.857 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.847+0000 7fa9e37fe640 10 monclient(hunting): handle_auth_done global_id 4129 payload 824 2024-03-13T07:33:50.857 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.847+0000 7fa9e37fe640 1 -- 172.21.15.195:0/3618394749 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9e413fd40 msgr2=0x7fa9e4138200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:50.857 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.847+0000 7fa9e37fe640 1 --2- 172.21.15.195:0/3618394749 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9e413fd40 0x7fa9e4138200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:50.857 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.847+0000 7fa9e37fe640 1 -- 172.21.15.195:0/3618394749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9e40b1ac0 msgr2=0x7fa9e41428a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:50.857 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.847+0000 7fa9e37fe640 1 --2- 172.21.15.195:0/3618394749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9e40b1ac0 0x7fa9e41428a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:50.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.847+0000 7fa9e37fe640 10 monclient: _finish_hunting 0 2024-03-13T07:33:50.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.847+0000 7fa9e37fe640 1 monclient: found mon.noname-b 2024-03-13T07:33:50.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.847+0000 7fa9e37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:50.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.847+0000 7fa9e37fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:33:50.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.847+0000 7fa9e37fe640 1 -- 172.21.15.195:0/3618394749 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e41464a0 con 0x7fa9e40b5320 2024-03-13T07:33:50.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.849+0000 7fa9e37fe640 1 --2- 172.21.15.195:0/3618394749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e40b5320 0x7fa9e4145f60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800c430 tx=0x7fa9d800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:50.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.849+0000 7fa9e0ff9640 10 client.?.objecter ms_handle_connect 0x7fa9e40b5320 2024-03-13T07:33:50.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.850+0000 7fa9e0ff9640 10 client.?.objecter resend_mon_ops 2024-03-13T07:33:50.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9e0ff9640 1 -- 172.21.15.195:0/3618394749 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d80052d0 con 0x7fa9e40b5320 2024-03-13T07:33:50.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9e0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:50.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9e0ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:33:50.858 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9e0ff9640 10 monclient: dump: 2024-03-13T07:33:50.858 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:50.858 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:50.859 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:50.859 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:50.859 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:50.859 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:50.859 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:50.859 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:50.859 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:50.859 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:50.859 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9e0ff9640 10 monclient: _finish_auth 0 2024-03-13T07:33:50.859 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9e0ff9640 10 monclient: _check_auth_tickets 2024-03-13T07:33:50.859 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9e0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:50.859 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9e0ff9640 1 -- 172.21.15.195:0/3618394749 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d8005490 con 0x7fa9e40b5320 2024-03-13T07:33:50.859 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9e0ff9640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:50.859 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9e0ff9640 1 -- 172.21.15.195:0/3618394749 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8007260 con 0x7fa9e40b5320 2024-03-13T07:33:50.859 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9ea5c5640 5 monclient: authenticate success, global_id 4129 2024-03-13T07:33:50.859 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9ea5c5640 10 monclient: _renew_subs 2024-03-13T07:33:50.860 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9ea5c5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:50.860 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9ea5c5640 1 -- 172.21.15.195:0/3618394749 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e41459f0 con 0x7fa9e40b5320 2024-03-13T07:33:50.860 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9e0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:50.860 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9e0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:33:50.860 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9e0ff9640 10 monclient: dump: 2024-03-13T07:33:50.860 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:50.860 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:50.860 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:50.860 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:50.860 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:50.860 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:50.860 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:50.860 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:50.860 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:50.860 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:50.861 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.853+0000 7fa9e27fc640 4 set_mon_vals no callback set 2024-03-13T07:33:50.861 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.854+0000 7fa9ea5c5640 10 client.4129.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:33:50.861 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.854+0000 7fa9ea5c5640 10 monclient: _renew_subs 2024-03-13T07:33:50.861 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.854+0000 7fa9ea5c5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:50.861 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.854+0000 7fa9ea5c5640 1 -- 172.21.15.195:0/3618394749 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e4138920 con 0x7fa9e40b5320 2024-03-13T07:33:50.861 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.854+0000 7fa9ea5c5640 1 librados: init done 2024-03-13T07:33:50.861 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.854+0000 7fa9ea5c5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:50.861 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.854+0000 7fa9ea5c5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:50.861 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.854+0000 7fa9ea5c5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:50.861 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.854+0000 7fa9ea5c5640 1 -- 172.21.15.195:0/3618394749 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e404f000 con 0x7fa9e40b5320 2024-03-13T07:33:50.861 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.857+0000 7fa9e0ff9640 1 -- 172.21.15.195:0/3618394749 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa9d8007400 con 0x7fa9e40b5320 2024-03-13T07:33:50.861 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.857+0000 7fa9e0ff9640 1 -- 172.21.15.195:0/3618394749 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fa9d8011590 con 0x7fa9e40b5320 2024-03-13T07:33:50.861 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.857+0000 7fa9e0ff9640 10 client.4129.objecter ms_dispatch 0x7fa9e4000b90 osd_map(4..4 src has 1..4) v4 2024-03-13T07:33:50.861 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.857+0000 7fa9e0ff9640 3 client.4129.objecter handle_osd_map got epochs [4,4] > 0 2024-03-13T07:33:50.862 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.857+0000 7fa9e0ff9640 3 client.4129.objecter handle_osd_map decoding full epoch 4 2024-03-13T07:33:50.862 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.857+0000 7fa9e0ff9640 20 client.4129.objecter dump_active .. 0 homeless 2024-03-13T07:33:50.902 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:50.902+0000 7fb6b5844040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-13T07:33:50.917 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:50.917+0000 7fd30663e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-13T07:33:50.947 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:50.947+0000 7fb6b5844040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-13T07:33:50.954 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.954+0000 7fa9e0ff9640 1 -- 172.21.15.195:0/3618394749 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa9d8003910 con 0x7fa9e40b5320 2024-03-13T07:33:50.954 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.954+0000 7fa9e0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:50.955 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:50.954+0000 7fa9e0ff9640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:33:50.963 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:50.963+0000 7fd30663e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-13T07:33:51.028 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.027+0000 7fa9ea5c5640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "18008b7f-cfe7-4450-a3cf-aab63ebbe99d", "id": 2}] 2024-03-13T07:33:51.028 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.027+0000 7fa9ea5c5640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "18008b7f-cfe7-4450-a3cf-aab63ebbe99d", "id": 2}] 2024-03-13T07:33:51.028 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.027+0000 7fa9ea5c5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:51.028 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.027+0000 7fa9ea5c5640 1 -- 172.21.15.195:0/3618394749 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "18008b7f-cfe7-4450-a3cf-aab63ebbe99d", "id": 2} v 0) v1 -- 0x7fa9e4138b80 con 0x7fa9e40b5320 2024-03-13T07:33:51.060 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:51.061+0000 7fb6b5844040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-13T07:33:51.080 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:51.080+0000 7fd30663e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-13T07:33:51.109 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.108+0000 7fa9e0ff9640 1 -- 172.21.15.195:0/3618394749 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "18008b7f-cfe7-4450-a3cf-aab63ebbe99d", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa9d8003c20 con 0x7fa9e40b5320 2024-03-13T07:33:51.109 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.108+0000 7fa9e0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "18008b7f-cfe7-4450-a3cf-aab63ebbe99d", "id": 2}] 2024-03-13T07:33:51.109 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.108+0000 7fa9e0ff9640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:33:51.109 INFO:teuthology.orchestra.run.smithi195.stdout:2 2024-03-13T07:33:51.111 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.109+0000 7fa9ea5c5640 10 librados: watch_flush enter 2024-03-13T07:33:51.111 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.109+0000 7fa9ea5c5640 10 librados: watch_flush exit 2024-03-13T07:33:51.111 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.109+0000 7fa9ea5c5640 20 client.4129.objecter shutdown clearing up homeless session... 2024-03-13T07:33:51.111 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.109+0000 7fa9ea5c5640 10 client.4129.objecter successfully canceled tick 2024-03-13T07:33:51.111 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.109+0000 7fa9ea5c5640 10 monclient: shutdown 2024-03-13T07:33:51.111 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.109+0000 7fa9ea5c5640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:51.111 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.109+0000 7fa9ea5c5640 1 -- 172.21.15.195:0/3618394749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e40b5320 msgr2=0x7fa9e4145f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:51.112 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.109+0000 7fa9ea5c5640 1 --2- 172.21.15.195:0/3618394749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e40b5320 0x7fa9e4145f60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800c430 tx=0x7fa9d800c8f0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.112 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.109+0000 7fa9ea5c5640 1 -- 172.21.15.195:0/3618394749 shutdown_connections 2024-03-13T07:33:51.112 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.109+0000 7fa9ea5c5640 1 --2- 172.21.15.195:0/3618394749 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9e413fd40 0x7fa9e4138200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.112 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.109+0000 7fa9ea5c5640 1 --2- 172.21.15.195:0/3618394749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e40b5320 0x7fa9e4145f60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.112 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.109+0000 7fa9ea5c5640 1 --2- 172.21.15.195:0/3618394749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9e40b1ac0 0x7fa9e41428a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.112 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.109+0000 7fa9ea5c5640 1 -- 172.21.15.195:0/3618394749 >> 172.21.15.195:0/3618394749 conn(0x7fa9e406d2b0 msgr2=0x7fa9e413f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:51.112 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.109+0000 7fa9ea5c5640 1 -- 172.21.15.195:0/3618394749 shutdown_connections 2024-03-13T07:33:51.112 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.109+0000 7fa9ea5c5640 1 -- 172.21.15.195:0/3618394749 wait complete. 2024-03-13T07:33:51.112 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.109+0000 7fa9ea5c5640 1 librados: shutdown 2024-03-13T07:33:51.120 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 411b6066-4b9f-40e7-b92d-4ba914eab63e 3 2024-03-13T07:33:51.158 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:51.158+0000 7fb6b5844040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-13T07:33:51.182 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:51.182+0000 7fd30663e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-13T07:33:51.220 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.220+0000 7f06673b7640 10 monclient: get_monmap_and_config 2024-03-13T07:33:51.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.220+0000 7f06673b7640 10 monclient: build_initial_monmap 2024-03-13T07:33:51.221 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.220+0000 7f06673b7640 10 monclient: monmap: 2024-03-13T07:33:51.221 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 0 2024-03-13T07:33:51.221 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:51.221 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:51.221348+0000 2024-03-13T07:33:51.221 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:51.221348+0000 2024-03-13T07:33:51.221 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:51.221 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:51.221 INFO:teuthology.orchestra.run.smithi195.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:33:51.222 INFO:teuthology.orchestra.run.smithi195.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:33:51.222 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:33:51.222 INFO:teuthology.orchestra.run.smithi195.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:33:51.222 INFO:teuthology.orchestra.run.smithi195.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:33:51.222 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:51.222 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.220+0000 7f06673b7640 1 Processor -- start 2024-03-13T07:33:51.222 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 1 -- start start 2024-03-13T07:33:51.222 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 10 monclient: init 2024-03-13T07:33:51.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:51.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 10 monclient: _add_conns ranks=[4,2,3,1,0] 2024-03-13T07:33:51.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f06600b5320 0x7f06600b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 10 monclient(hunting): picked mon.noname-c con 0x7f06600b5320 addr v2:172.21.15.195:3300/0 2024-03-13T07:33:51.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06600b1ac0 0x7f06600b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 10 monclient(hunting): picked mon.noname-b con 0x7f06600b1ac0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:51.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f066013f630 0x7f066013fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 10 monclient(hunting): picked mon.noname-a con 0x7f066013f630 addr v2:172.21.15.132:3300/0 2024-03-13T07:33:51.225 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.225 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06600b5700 con 0x7f066013f630 2024-03-13T07:33:51.225 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.225 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06600b1150 con 0x7f06600b5320 2024-03-13T07:33:51.225 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.225 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066013ff80 con 0x7f06600b1ac0 2024-03-13T07:33:51.225 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:51.225 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.221+0000 7f06673b7640 10 monclient(hunting): authenticate will time out at 1245.560070s 2024-03-13T07:33:51.225 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.222+0000 7f0665954640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f066013f630 0x7f066013fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.225 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.223+0000 7f0665954640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f066013f630 0x7f066013fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.195:52620/0 (socket says 172.21.15.195:52620) 2024-03-13T07:33:51.225 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.223+0000 7f0665954640 1 -- 172.21.15.195:0/1072667264 learned_addr learned my addr 172.21.15.195:0/1072667264 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-13T07:33:51.225 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.223+0000 7f0665954640 10 monclient(hunting): get_auth_request con 0x7f066013f630 auth_method 0 2024-03-13T07:33:51.226 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.223+0000 7f0665954640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:51.226 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.223+0000 7f0665954640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:51.226 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.223+0000 7f0665954640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:51.226 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.223+0000 7f0664952640 1 --2- 172.21.15.195:0/1072667264 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06600b1ac0 0x7f06600b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.224+0000 7f0664952640 10 monclient(hunting): get_auth_request con 0x7f06600b1ac0 auth_method 0 2024-03-13T07:33:51.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.224+0000 7f0664952640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:51.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.224+0000 7f0664952640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:51.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.224+0000 7f0664952640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:51.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.225+0000 7f0664952640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:51.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.225+0000 7f0664952640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:51.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.225+0000 7f0664952640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:51.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.227+0000 7f0664952640 10 monclient(hunting): handle_auth_done global_id 4118 payload 274 2024-03-13T07:33:51.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.227+0000 7f0664952640 1 -- 172.21.15.195:0/1072667264 >> v2:172.21.15.195:3300/0 conn(0x7f06600b5320 msgr2=0x7f06600b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:33:51.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.227+0000 7f0665153640 1 --2- 172.21.15.195:0/1072667264 >> v2:172.21.15.195:3300/0 conn(0x7f06600b5320 0x7f06600b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.227+0000 7f0664952640 1 --2- 172.21.15.195:0/1072667264 >> v2:172.21.15.195:3300/0 conn(0x7f06600b5320 0x7f06600b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.228 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.227+0000 7f0664952640 1 -- 172.21.15.195:0/1072667264 >> v2:172.21.15.132:3300/0 conn(0x7f066013f630 msgr2=0x7f066013fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:33:51.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.227+0000 7f0664952640 1 --2- 172.21.15.195:0/1072667264 >> v2:172.21.15.132:3300/0 conn(0x7f066013f630 0x7f066013fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.227+0000 7f0664952640 10 monclient: _finish_hunting 0 2024-03-13T07:33:51.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.227+0000 7f0664952640 1 monclient: found mon.noname-b 2024-03-13T07:33:51.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.227+0000 7f0664952640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:51.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.227+0000 7f0664952640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T07:33:51.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.227+0000 7f0664952640 1 -- 172.21.15.195:0/1072667264 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06600b13c0 con 0x7f06600b1ac0 2024-03-13T07:33:51.245 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.245+0000 7f0664952640 1 --2- 172.21.15.195:0/1072667264 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06600b1ac0 0x7f06600b1f00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f06600b90f0 tx=0x7f065002b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99416b9dc6fd5e6b server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:51.251 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.250+0000 7f0657fff640 1 -- 172.21.15.195:0/1072667264 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650037070 con 0x7f06600b1ac0 2024-03-13T07:33:51.251 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.250+0000 7f0657fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:51.251 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.250+0000 7f0657fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:33:51.251 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.250+0000 7f0657fff640 10 monclient: dump: 2024-03-13T07:33:51.251 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:51.251 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:51.252 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.252 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.252 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:51.252 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:51.252 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:51.252 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:51.252 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:51.252 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:51.252 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.250+0000 7f0657fff640 10 monclient: _finish_auth 0 2024-03-13T07:33:51.252 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.250+0000 7f0657fff640 10 monclient: _check_auth_tickets 2024-03-13T07:33:51.252 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.250+0000 7f0657fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:51.252 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.250+0000 7f0657fff640 1 -- 172.21.15.195:0/1072667264 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0650030710 con 0x7f06600b1ac0 2024-03-13T07:33:51.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.250+0000 7f0657fff640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:51.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.250+0000 7f0657fff640 1 -- 172.21.15.195:0/1072667264 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650030ae0 con 0x7f06600b1ac0 2024-03-13T07:33:51.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.250+0000 7f06673b7640 5 monclient: authenticate success, global_id 4118 2024-03-13T07:33:51.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.250+0000 7f06673b7640 10 monclient: get_monmap_and_config success 2024-03-13T07:33:51.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.250+0000 7f06673b7640 4 set_mon_vals no callback set 2024-03-13T07:33:51.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.250+0000 7f0657fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:51.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.250+0000 7f0657fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:33:51.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.250+0000 7f0657fff640 10 monclient: dump: 2024-03-13T07:33:51.253 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:51.253 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:51.253 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.254 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.254 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:51.254 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:51.254 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:51.254 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:51.254 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:51.254 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:51.259 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.251+0000 7f06673b7640 10 monclient: shutdown 2024-03-13T07:33:51.259 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.251+0000 7f06673b7640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:51.259 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.251+0000 7f06673b7640 1 -- 172.21.15.195:0/1072667264 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06600b1ac0 msgr2=0x7f06600b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:51.260 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.251+0000 7f06673b7640 1 --2- 172.21.15.195:0/1072667264 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06600b1ac0 0x7f06600b1f00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f06600b90f0 tx=0x7f065002b710 comp rx=0 tx=0).stop 2024-03-13T07:33:51.260 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.251+0000 7f06673b7640 1 -- 172.21.15.195:0/1072667264 shutdown_connections 2024-03-13T07:33:51.260 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.251+0000 7f06673b7640 1 --2- 172.21.15.195:0/1072667264 >> v2:172.21.15.132:3300/0 conn(0x7f066013f630 0x7f066013fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.260 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.251+0000 7f06673b7640 1 --2- 172.21.15.195:0/1072667264 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06600b1ac0 0x7f06600b1f00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.260 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.251+0000 7f06673b7640 1 --2- 172.21.15.195:0/1072667264 >> v2:172.21.15.195:3300/0 conn(0x7f06600b5320 0x7f06600b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.260 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.251+0000 7f06673b7640 1 -- 172.21.15.195:0/1072667264 >> 172.21.15.195:0/1072667264 conn(0x7f066006d2b0 msgr2=0x7f066006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:51.260 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.251+0000 7f06673b7640 1 -- 172.21.15.195:0/1072667264 shutdown_connections 2024-03-13T07:33:51.260 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.251+0000 7f06673b7640 1 -- 172.21.15.195:0/1072667264 wait complete. 2024-03-13T07:33:51.260 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.252+0000 7f06673b7640 10 monclient: build_initial_monmap 2024-03-13T07:33:51.260 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.252+0000 7f06673b7640 10 monclient: monmap: 2024-03-13T07:33:51.260 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 0 2024-03-13T07:33:51.261 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:33:51.261 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 0.000000 2024-03-13T07:33:51.261 INFO:teuthology.orchestra.run.smithi195.stderr:created 0.000000 2024-03-13T07:33:51.261 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:51.261 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:51.261 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:33:51.261 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:33:51.261 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:33:51.261 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:51.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.252+0000 7f06673b7640 1 librados: starting msgr at 2024-03-13T07:33:51.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.252+0000 7f06673b7640 1 librados: starting objecter 2024-03-13T07:33:51.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.252+0000 7f06673b7640 1 Processor -- start 2024-03-13T07:33:51.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 1 -- start start 2024-03-13T07:33:51.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 1 librados: setting wanted keys 2024-03-13T07:33:51.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 1 librados: calling monclient init 2024-03-13T07:33:51.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 10 monclient: init 2024-03-13T07:33:51.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:51.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 10 monclient: _add_conns ranks=[2,0,1] 2024-03-13T07:33:51.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06600b1ac0 0x7f06601d87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 10 monclient(hunting): picked mon.noname-c con 0x7f06600b1ac0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:51.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06600b5320 0x7f06600ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.263 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 10 monclient(hunting): picked mon.noname-a con 0x7f06600b5320 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:33:51.263 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f066013f630 0x7f06601429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.263 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 10 monclient(hunting): picked mon.noname-b con 0x7f066013f630 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:33:51.263 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.263 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06600b4800 con 0x7f06600b5320 2024-03-13T07:33:51.263 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.263 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06600afd00 con 0x7f066013f630 2024-03-13T07:33:51.263 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.263 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06600b4980 con 0x7f06600b1ac0 2024-03-13T07:33:51.263 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:51.263 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.253+0000 7f06673b7640 10 monclient(hunting): authenticate will time out at 1245.592003s 2024-03-13T07:33:51.263 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.254+0000 7f0665153640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06600b1ac0 0x7f06601d87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.264 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.254+0000 7f0665954640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f066013f630 0x7f06601429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.264 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.254+0000 7f0665153640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06600b1ac0 0x7f06601d87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.195:48764/0 (socket says 172.21.15.195:48764) 2024-03-13T07:33:51.264 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.254+0000 7f0665153640 1 -- 172.21.15.195:0/1862253677 learned_addr learned my addr 172.21.15.195:0/1862253677 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-13T07:33:51.264 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.254+0000 7f0665153640 10 monclient(hunting): get_auth_request con 0x7f06600b1ac0 auth_method 0 2024-03-13T07:33:51.264 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.254+0000 7f0665153640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:51.264 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.254+0000 7f0665153640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:51.264 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.254+0000 7f0665153640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:51.264 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.254+0000 7f0665954640 10 monclient(hunting): get_auth_request con 0x7f066013f630 auth_method 0 2024-03-13T07:33:51.264 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.254+0000 7f0665954640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:51.264 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.254+0000 7f0665954640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:51.264 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.254+0000 7f0665954640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:51.265 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.254+0000 7f0664952640 1 --2- 172.21.15.195:0/1862253677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06600b5320 0x7f06600ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.265 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.255+0000 7f0665954640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:51.265 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.255+0000 7f0665954640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:51.265 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.255+0000 7f0665954640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:51.265 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.255+0000 7f0665153640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:51.265 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.255+0000 7f0665153640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:51.265 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.255+0000 7f0665153640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:51.265 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.255+0000 7f0664952640 10 monclient(hunting): get_auth_request con 0x7f06600b5320 auth_method 0 2024-03-13T07:33:51.265 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.255+0000 7f0664952640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:51.265 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.255+0000 7f0664952640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:51.265 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.255+0000 7f0664952640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:51.266 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.255+0000 7f0664952640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:51.266 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.255+0000 7f0664952640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:51.266 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.255+0000 7f0664952640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:51.266 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.257+0000 7f0665954640 10 monclient(hunting): handle_auth_done global_id 4132 payload 824 2024-03-13T07:33:51.266 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.257+0000 7f0665954640 1 -- 172.21.15.195:0/1862253677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06600b1ac0 msgr2=0x7f06601d87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:51.266 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.258+0000 7f0665954640 1 --2- 172.21.15.195:0/1862253677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06600b1ac0 0x7f06601d87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.266 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.258+0000 7f0665954640 1 -- 172.21.15.195:0/1862253677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06600b5320 msgr2=0x7f06600ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:51.266 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.258+0000 7f0665954640 1 --2- 172.21.15.195:0/1862253677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06600b5320 0x7f06600ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.266 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.258+0000 7f0665954640 10 monclient: _finish_hunting 0 2024-03-13T07:33:51.266 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.258+0000 7f0665954640 1 monclient: found mon.noname-b 2024-03-13T07:33:51.266 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.258+0000 7f0665954640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:51.266 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.258+0000 7f0665954640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:33:51.267 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.258+0000 7f0665954640 1 -- 172.21.15.195:0/1862253677 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06601d82c0 con 0x7f066013f630 2024-03-13T07:33:51.268 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.260+0000 7f0665954640 1 --2- 172.21.15.195:0/1862253677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f066013f630 0x7f06601429c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0658007a90 tx=0x7f0658007e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:51.269 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.260+0000 7f0655ffb640 10 client.?.objecter ms_handle_connect 0x7f066013f630 2024-03-13T07:33:51.269 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.260+0000 7f0655ffb640 10 client.?.objecter resend_mon_ops 2024-03-13T07:33:51.269 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f0655ffb640 1 -- 172.21.15.195:0/1862253677 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065801a070 con 0x7f066013f630 2024-03-13T07:33:51.269 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f0655ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:51.269 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f0655ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:33:51.269 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f0655ffb640 10 monclient: dump: 2024-03-13T07:33:51.269 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:51.269 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:51.269 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.269 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.269 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:51.269 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:51.270 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:51.270 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:51.270 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:51.270 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:51.270 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f0655ffb640 10 monclient: _finish_auth 0 2024-03-13T07:33:51.270 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f0655ffb640 10 monclient: _check_auth_tickets 2024-03-13T07:33:51.270 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f0655ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:51.270 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f0655ffb640 1 -- 172.21.15.195:0/1862253677 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065800dbe0 con 0x7f066013f630 2024-03-13T07:33:51.270 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f0655ffb640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:51.270 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f0655ffb640 1 -- 172.21.15.195:0/1862253677 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658015400 con 0x7f066013f630 2024-03-13T07:33:51.270 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f06673b7640 5 monclient: authenticate success, global_id 4132 2024-03-13T07:33:51.271 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f0655ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:51.271 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f0655ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:33:51.271 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f0655ffb640 10 monclient: dump: 2024-03-13T07:33:51.271 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:51.271 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:51.271 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.271 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.271 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:51.271 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:51.271 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:51.271 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:51.272 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:51.272 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:51.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f06577fe640 4 set_mon_vals no callback set 2024-03-13T07:33:51.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f06673b7640 10 monclient: _renew_subs 2024-03-13T07:33:51.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f06673b7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:51.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f06673b7640 1 -- 172.21.15.195:0/1862253677 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06601d7c90 con 0x7f066013f630 2024-03-13T07:33:51.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.264+0000 7f06673b7640 10 client.4132.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:33:51.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.265+0000 7f06673b7640 10 monclient: _renew_subs 2024-03-13T07:33:51.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.265+0000 7f06673b7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:51.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.265+0000 7f06673b7640 1 -- 172.21.15.195:0/1862253677 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0660143960 con 0x7f066013f630 2024-03-13T07:33:51.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.265+0000 7f06673b7640 1 librados: init done 2024-03-13T07:33:51.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.265+0000 7f06673b7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:51.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.265+0000 7f06673b7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:51.272 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.265+0000 7f06673b7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:51.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.266+0000 7f06673b7640 1 -- 172.21.15.195:0/1862253677 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066004f000 con 0x7f066013f630 2024-03-13T07:33:51.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.268+0000 7f0655ffb640 1 -- 172.21.15.195:0/1862253677 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f065801d070 con 0x7f066013f630 2024-03-13T07:33:51.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.268+0000 7f0655ffb640 1 -- 172.21.15.195:0/1862253677 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f065801e840 con 0x7f066013f630 2024-03-13T07:33:51.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.268+0000 7f0655ffb640 10 client.4132.objecter ms_dispatch 0x7f0660000b90 osd_map(5..5 src has 1..5) v4 2024-03-13T07:33:51.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.268+0000 7f0655ffb640 3 client.4132.objecter handle_osd_map got epochs [5,5] > 0 2024-03-13T07:33:51.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.268+0000 7f0655ffb640 3 client.4132.objecter handle_osd_map decoding full epoch 5 2024-03-13T07:33:51.273 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.268+0000 7f0655ffb640 20 client.4132.objecter dump_active .. 0 homeless 2024-03-13T07:33:51.318 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:51.318+0000 7fb6b5844040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-13T07:33:51.351 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:51.351+0000 7fd30663e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-13T07:33:51.360 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:51.360+0000 7fb6b5844040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-13T07:33:51.368 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.368+0000 7f0655ffb640 1 -- 172.21.15.195:0/1862253677 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0658024020 con 0x7f066013f630 2024-03-13T07:33:51.369 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.368+0000 7f0655ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:51.369 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.368+0000 7f0655ffb640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:33:51.397 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:51.397+0000 7fd30663e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-13T07:33:51.443 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.442+0000 7f06673b7640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "411b6066-4b9f-40e7-b92d-4ba914eab63e", "id": 3}] 2024-03-13T07:33:51.443 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.442+0000 7f06673b7640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "411b6066-4b9f-40e7-b92d-4ba914eab63e", "id": 3}] 2024-03-13T07:33:51.443 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.442+0000 7f06673b7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:51.443 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.442+0000 7f06673b7640 1 -- 172.21.15.195:0/1862253677 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "411b6066-4b9f-40e7-b92d-4ba914eab63e", "id": 3} v 0) v1 -- 0x7f06601d8490 con 0x7f066013f630 2024-03-13T07:33:51.504 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.504+0000 7f0655ffb640 1 -- 172.21.15.195:0/1862253677 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "411b6066-4b9f-40e7-b92d-4ba914eab63e", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0658010070 con 0x7f066013f630 2024-03-13T07:33:51.504 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.504+0000 7f0655ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "411b6066-4b9f-40e7-b92d-4ba914eab63e", "id": 3}] 2024-03-13T07:33:51.505 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.504+0000 7f0655ffb640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:33:51.505 INFO:teuthology.orchestra.run.smithi195.stdout:3 2024-03-13T07:33:51.506 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.505+0000 7f06673b7640 10 librados: watch_flush enter 2024-03-13T07:33:51.506 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.505+0000 7f06673b7640 10 librados: watch_flush exit 2024-03-13T07:33:51.507 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.505+0000 7f06673b7640 20 client.4132.objecter shutdown clearing up homeless session... 2024-03-13T07:33:51.507 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.505+0000 7f06673b7640 10 client.4132.objecter successfully canceled tick 2024-03-13T07:33:51.507 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.505+0000 7f06673b7640 10 monclient: shutdown 2024-03-13T07:33:51.507 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.505+0000 7f06673b7640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:51.507 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.505+0000 7f06673b7640 1 -- 172.21.15.195:0/1862253677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f066013f630 msgr2=0x7f06601429c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:51.507 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.505+0000 7f06673b7640 1 --2- 172.21.15.195:0/1862253677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f066013f630 0x7f06601429c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0658007a90 tx=0x7f0658007e90 comp rx=0 tx=0).stop 2024-03-13T07:33:51.507 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.505+0000 7f06673b7640 1 -- 172.21.15.195:0/1862253677 shutdown_connections 2024-03-13T07:33:51.507 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.505+0000 7f06673b7640 1 --2- 172.21.15.195:0/1862253677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f066013f630 0x7f06601429c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.507 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.505+0000 7f06673b7640 1 --2- 172.21.15.195:0/1862253677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06600b5320 0x7f06600ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.507 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.505+0000 7f06673b7640 1 --2- 172.21.15.195:0/1862253677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06600b1ac0 0x7f06601d87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.507 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.505+0000 7f06673b7640 1 -- 172.21.15.195:0/1862253677 >> 172.21.15.195:0/1862253677 conn(0x7f066006d2b0 msgr2=0x7f066013b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:51.507 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.505+0000 7f06673b7640 1 -- 172.21.15.195:0/1862253677 shutdown_connections 2024-03-13T07:33:51.507 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.505+0000 7f06673b7640 1 -- 172.21.15.195:0/1862253677 wait complete. 2024-03-13T07:33:51.507 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.505+0000 7f06673b7640 1 librados: shutdown 2024-03-13T07:33:51.516 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 1a719246-ae04-42b0-bd9c-fc318dc57ecb 4 2024-03-13T07:33:51.524 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:51.525+0000 7fb6b5844040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-13T07:33:51.563 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:51.563+0000 7fd30663e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-13T07:33:51.618 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.615+0000 7ff8088d3640 10 monclient: get_monmap_and_config 2024-03-13T07:33:51.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.615+0000 7ff8088d3640 10 monclient: build_initial_monmap 2024-03-13T07:33:51.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.615+0000 7ff8088d3640 10 monclient: monmap: 2024-03-13T07:33:51.619 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 0 2024-03-13T07:33:51.619 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:51.619 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:51.616713+0000 2024-03-13T07:33:51.619 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:51.616713+0000 2024-03-13T07:33:51.619 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:51.619 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:51.619 INFO:teuthology.orchestra.run.smithi195.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:33:51.619 INFO:teuthology.orchestra.run.smithi195.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:33:51.619 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:33:51.620 INFO:teuthology.orchestra.run.smithi195.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:33:51.620 INFO:teuthology.orchestra.run.smithi195.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:33:51.620 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:51.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.616+0000 7ff8088d3640 1 Processor -- start 2024-03-13T07:33:51.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.616+0000 7ff8088d3640 1 -- start start 2024-03-13T07:33:51.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.616+0000 7ff8088d3640 10 monclient: init 2024-03-13T07:33:51.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.616+0000 7ff8088d3640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:51.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.616+0000 7ff8088d3640 10 monclient: _add_conns ranks=[4,3,2,0,1] 2024-03-13T07:33:51.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.616+0000 7ff8088d3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff804070bf0 0x7ff804070fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.616+0000 7ff8088d3640 10 monclient(hunting): picked mon.noname-c con 0x7ff804070bf0 addr v2:172.21.15.195:3300/0 2024-03-13T07:33:51.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.616+0000 7ff8088d3640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff804071d70 0x7ff8040b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.616+0000 7ff8088d3640 10 monclient(hunting): picked mon.noname-a con 0x7ff804071d70 addr v2:172.21.15.132:3300/0 2024-03-13T07:33:51.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.616+0000 7ff8088d3640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8040b1c00 0x7ff8040b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.617+0000 7ff8088d3640 10 monclient(hunting): picked mon.noname-b con 0x7ff8040b1c00 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:51.621 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.617+0000 7ff8088d3640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.621 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.617+0000 7ff8088d3640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff80413e0a0 con 0x7ff804071d70 2024-03-13T07:33:51.621 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.617+0000 7ff8088d3640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.621 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.617+0000 7ff8088d3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff804144160 con 0x7ff804070bf0 2024-03-13T07:33:51.621 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.617+0000 7ff8088d3640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.621 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.617+0000 7ff8088d3640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff804143e50 con 0x7ff8040b1c00 2024-03-13T07:33:51.621 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.617+0000 7ff8088d3640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:51.621 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.617+0000 7ff8088d3640 10 monclient(hunting): authenticate will time out at 1245.955415s 2024-03-13T07:33:51.621 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.617+0000 7ff80259c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff804070bf0 0x7ff804070fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.621 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.617+0000 7ff802d9d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8040b1c00 0x7ff8040b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.621 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.617+0000 7ff801d9b640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff804071d70 0x7ff8040b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.621 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff80259c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff804070bf0 0x7ff804070fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:49870/0 (socket says 172.21.15.195:49870) 2024-03-13T07:33:51.621 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff80259c640 1 -- 172.21.15.195:0/1351815851 learned_addr learned my addr 172.21.15.195:0/1351815851 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-13T07:33:51.622 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff80259c640 10 monclient(hunting): get_auth_request con 0x7ff804070bf0 auth_method 0 2024-03-13T07:33:51.622 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff80259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:51.622 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff80259c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:51.622 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff80259c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:51.622 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff802d9d640 10 monclient(hunting): get_auth_request con 0x7ff8040b1c00 auth_method 0 2024-03-13T07:33:51.622 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff802d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:51.622 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff802d9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:51.622 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff802d9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:51.622 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff801d9b640 10 monclient(hunting): get_auth_request con 0x7ff804071d70 auth_method 0 2024-03-13T07:33:51.622 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff801d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:51.622 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff801d9b640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:51.622 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff801d9b640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:51.622 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff80259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:51.622 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff80259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:51.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff80259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:51.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff802d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:51.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff802d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:51.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.618+0000 7ff802d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:51.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.619+0000 7ff801d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:51.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.619+0000 7ff801d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:51.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.619+0000 7ff801d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:51.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.620+0000 7ff80259c640 10 monclient(hunting): handle_auth_done global_id 4135 payload 274 2024-03-13T07:33:51.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.620+0000 7ff80259c640 1 -- 172.21.15.195:0/1351815851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8040b1c00 msgr2=0x7ff8040b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:33:51.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.620+0000 7ff80259c640 1 --2- 172.21.15.195:0/1351815851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8040b1c00 0x7ff8040b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.620+0000 7ff80259c640 1 -- 172.21.15.195:0/1351815851 >> v2:172.21.15.132:3300/0 conn(0x7ff804071d70 msgr2=0x7ff8040b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:33:51.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.620+0000 7ff80259c640 1 --2- 172.21.15.195:0/1351815851 >> v2:172.21.15.132:3300/0 conn(0x7ff804071d70 0x7ff8040b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.620+0000 7ff80259c640 10 monclient: _finish_hunting 0 2024-03-13T07:33:51.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.620+0000 7ff80259c640 1 monclient: found mon.noname-c 2024-03-13T07:33:51.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.620+0000 7ff80259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:51.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.620+0000 7ff80259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T07:33:51.625 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.620+0000 7ff80259c640 1 -- 172.21.15.195:0/1351815851 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff804070270 con 0x7ff804070bf0 2024-03-13T07:33:51.625 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.621+0000 7ff80259c640 1 --2- 172.21.15.195:0/1351815851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff804070bf0 0x7ff804070fd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff8040b20e0 tx=0x7ff7f402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afd2b6af26671d20 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:51.625 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff80159a640 1 -- 172.21.15.195:0/1351815851 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4035020 con 0x7ff804070bf0 2024-03-13T07:33:51.626 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff80159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:51.626 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff80159a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:33:51.626 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff80159a640 10 monclient: dump: 2024-03-13T07:33:51.626 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:51.626 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:51.626 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.626 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.626 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:51.626 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:51.626 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:51.626 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:51.626 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:51.626 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:51.626 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff80159a640 10 monclient: _finish_auth 0 2024-03-13T07:33:51.627 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff80159a640 10 monclient: _check_auth_tickets 2024-03-13T07:33:51.627 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff80159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:51.627 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff8088d3640 5 monclient: authenticate success, global_id 4135 2024-03-13T07:33:51.627 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff80159a640 1 -- 172.21.15.195:0/1351815851 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f4031870 con 0x7ff804070bf0 2024-03-13T07:33:51.627 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff8088d3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:33:51.627 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff80159a640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:51.627 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff8088d3640 10 monclient: get_monmap_and_config success 2024-03-13T07:33:51.627 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff8088d3640 4 set_mon_vals no callback set 2024-03-13T07:33:51.627 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff80159a640 1 -- 172.21.15.195:0/1351815851 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4031c40 con 0x7ff804070bf0 2024-03-13T07:33:51.627 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff80159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:51.627 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff80159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:33:51.627 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.625+0000 7ff80159a640 10 monclient: dump: 2024-03-13T07:33:51.628 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:51.628 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:51.628 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.628 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.628 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:51.628 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:51.628 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:51.628 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:51.628 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:51.628 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:51.632 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.626+0000 7ff8088d3640 10 monclient: shutdown 2024-03-13T07:33:51.632 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.626+0000 7ff8088d3640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:51.632 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.626+0000 7ff8088d3640 1 -- 172.21.15.195:0/1351815851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff804070bf0 msgr2=0x7ff804070fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:51.632 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.626+0000 7ff8088d3640 1 --2- 172.21.15.195:0/1351815851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff804070bf0 0x7ff804070fd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff8040b20e0 tx=0x7ff7f402b910 comp rx=0 tx=0).stop 2024-03-13T07:33:51.632 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.626+0000 7ff8088d3640 1 -- 172.21.15.195:0/1351815851 shutdown_connections 2024-03-13T07:33:51.632 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.626+0000 7ff8088d3640 1 --2- 172.21.15.195:0/1351815851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8040b1c00 0x7ff8040b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.632 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.626+0000 7ff8088d3640 1 --2- 172.21.15.195:0/1351815851 >> v2:172.21.15.132:3300/0 conn(0x7ff804071d70 0x7ff8040b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.632 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.626+0000 7ff8088d3640 1 --2- 172.21.15.195:0/1351815851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff804070bf0 0x7ff804070fd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.632 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.626+0000 7ff8088d3640 1 -- 172.21.15.195:0/1351815851 >> 172.21.15.195:0/1351815851 conn(0x7ff8040b6050 msgr2=0x7ff8040b6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:51.632 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.626+0000 7ff8088d3640 1 -- 172.21.15.195:0/1351815851 shutdown_connections 2024-03-13T07:33:51.633 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.626+0000 7ff8088d3640 1 -- 172.21.15.195:0/1351815851 wait complete. 2024-03-13T07:33:51.633 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.627+0000 7ff8088d3640 10 monclient: build_initial_monmap 2024-03-13T07:33:51.633 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.627+0000 7ff8088d3640 10 monclient: monmap: 2024-03-13T07:33:51.633 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 0 2024-03-13T07:33:51.633 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:33:51.633 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 0.000000 2024-03-13T07:33:51.633 INFO:teuthology.orchestra.run.smithi195.stderr:created 0.000000 2024-03-13T07:33:51.633 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:51.633 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:51.633 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:33:51.633 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:33:51.633 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:33:51.633 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:51.633 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.627+0000 7ff8088d3640 1 librados: starting msgr at 2024-03-13T07:33:51.633 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.627+0000 7ff8088d3640 1 librados: starting objecter 2024-03-13T07:33:51.634 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 1 Processor -- start 2024-03-13T07:33:51.634 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 1 -- start start 2024-03-13T07:33:51.634 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 1 librados: setting wanted keys 2024-03-13T07:33:51.634 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 1 librados: calling monclient init 2024-03-13T07:33:51.634 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 10 monclient: init 2024-03-13T07:33:51.634 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:51.634 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 10 monclient: _add_conns ranks=[0,1,2] 2024-03-13T07:33:51.634 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff804070bf0 0x7ff80413c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.634 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 10 monclient(hunting): picked mon.noname-a con 0x7ff804070bf0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:33:51.634 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff804071d70 0x7ff8041db330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.634 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 10 monclient(hunting): picked mon.noname-b con 0x7ff804071d70 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:33:51.634 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8040b1c00 0x7ff8041db870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.634 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 10 monclient(hunting): picked mon.noname-c con 0x7ff8040b1c00 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:51.634 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.635 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8040b0db0 con 0x7ff804070bf0 2024-03-13T07:33:51.635 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.635 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff804144840 con 0x7ff804071d70 2024-03-13T07:33:51.635 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.635 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8040b0f30 con 0x7ff8040b1c00 2024-03-13T07:33:51.635 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:51.635 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.628+0000 7ff8088d3640 10 monclient(hunting): authenticate will time out at 1245.967032s 2024-03-13T07:33:51.635 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff801d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff804071d70 0x7ff8041db330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.635 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff802d9d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8040b1c00 0x7ff8041db870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.635 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff80259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff804070bf0 0x7ff80413c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.635 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff801d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff804071d70 0x7ff8041db330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:49882/0 (socket says 172.21.15.195:49882) 2024-03-13T07:33:51.635 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff801d9b640 1 -- 172.21.15.195:0/2430945880 learned_addr learned my addr 172.21.15.195:0/2430945880 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-13T07:33:51.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff801d9b640 10 monclient(hunting): get_auth_request con 0x7ff804071d70 auth_method 0 2024-03-13T07:33:51.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff801d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:51.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff801d9b640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:51.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff801d9b640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:51.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff802d9d640 10 monclient(hunting): get_auth_request con 0x7ff8040b1c00 auth_method 0 2024-03-13T07:33:51.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff802d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:51.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff802d9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:51.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff802d9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:51.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff80259c640 10 monclient(hunting): get_auth_request con 0x7ff804070bf0 auth_method 0 2024-03-13T07:33:51.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff80259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:51.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff80259c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:51.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.629+0000 7ff80259c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:51.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.630+0000 7ff801d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:51.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.630+0000 7ff801d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:51.637 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.630+0000 7ff801d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:51.637 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.630+0000 7ff802d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:51.637 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.630+0000 7ff802d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:51.637 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.630+0000 7ff802d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:51.637 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.630+0000 7ff80259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:51.637 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.630+0000 7ff80259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:51.637 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.630+0000 7ff80259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:51.639 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.632+0000 7ff801d9b640 10 monclient(hunting): handle_auth_done global_id 4138 payload 824 2024-03-13T07:33:51.639 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.632+0000 7ff801d9b640 1 -- 172.21.15.195:0/2430945880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8040b1c00 msgr2=0x7ff8041db870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:51.640 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.632+0000 7ff801d9b640 1 --2- 172.21.15.195:0/2430945880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8040b1c00 0x7ff8041db870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.640 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.632+0000 7ff801d9b640 1 -- 172.21.15.195:0/2430945880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff804070bf0 msgr2=0x7ff80413c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:51.640 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.632+0000 7ff801d9b640 1 --2- 172.21.15.195:0/2430945880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff804070bf0 0x7ff80413c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.640 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.632+0000 7ff801d9b640 10 monclient: _finish_hunting 0 2024-03-13T07:33:51.640 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.632+0000 7ff801d9b640 1 monclient: found mon.noname-b 2024-03-13T07:33:51.640 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.632+0000 7ff801d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:51.640 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.632+0000 7ff801d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:33:51.640 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.632+0000 7ff801d9b640 1 -- 172.21.15.195:0/2430945880 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8040a9f40 con 0x7ff804071d70 2024-03-13T07:33:51.640 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.634+0000 7ff801d9b640 1 --2- 172.21.15.195:0/2430945880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff804071d70 0x7ff8041db330 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800e9b0 tx=0x7ff7f800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:51.640 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.634+0000 7ff7f2ffd640 10 client.?.objecter ms_handle_connect 0x7ff804071d70 2024-03-13T07:33:51.640 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.634+0000 7ff7f2ffd640 10 client.?.objecter resend_mon_ops 2024-03-13T07:33:51.640 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff7f2ffd640 1 -- 172.21.15.195:0/2430945880 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8017070 con 0x7ff804071d70 2024-03-13T07:33:51.640 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff7f2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:51.640 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff7f2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:33:51.640 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff7f2ffd640 10 monclient: dump: 2024-03-13T07:33:51.641 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:51.641 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:51.641 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.641 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.641 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:51.641 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:51.641 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:51.641 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:51.641 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:51.641 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:51.641 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff7f2ffd640 10 monclient: _finish_auth 0 2024-03-13T07:33:51.641 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff7f2ffd640 10 monclient: _check_auth_tickets 2024-03-13T07:33:51.641 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff7f2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:51.641 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff7f2ffd640 1 -- 172.21.15.195:0/2430945880 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f800c9a0 con 0x7ff804071d70 2024-03-13T07:33:51.641 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff8088d3640 5 monclient: authenticate success, global_id 4138 2024-03-13T07:33:51.641 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff7f2ffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:51.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff8088d3640 10 monclient: _renew_subs 2024-03-13T07:33:51.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff7f2ffd640 1 -- 172.21.15.195:0/2430945880 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800cd90 con 0x7ff804071d70 2024-03-13T07:33:51.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff8088d3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:51.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff8088d3640 1 -- 172.21.15.195:0/2430945880 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8040aa230 con 0x7ff804071d70 2024-03-13T07:33:51.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff7f2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:51.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff800d99640 4 set_mon_vals no callback set 2024-03-13T07:33:51.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff7f2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:33:51.642 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff7f2ffd640 10 monclient: dump: 2024-03-13T07:33:51.642 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:51.642 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:51.642 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.642 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.642 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:51.643 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:51.643 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:51.643 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:51.643 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:51.643 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:51.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff8088d3640 10 client.4138.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:33:51.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff8088d3640 10 monclient: _renew_subs 2024-03-13T07:33:51.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff8088d3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:51.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff8088d3640 1 -- 172.21.15.195:0/2430945880 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8040aa780 con 0x7ff804071d70 2024-03-13T07:33:51.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.638+0000 7ff8088d3640 1 librados: init done 2024-03-13T07:33:51.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.639+0000 7ff8088d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:51.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.639+0000 7ff8088d3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:51.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.639+0000 7ff8088d3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:51.643 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.639+0000 7ff8088d3640 1 -- 172.21.15.195:0/2430945880 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8041dbf70 con 0x7ff804071d70 2024-03-13T07:33:51.644 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.642+0000 7ff7f2ffd640 1 -- 172.21.15.195:0/2430945880 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff7f801a070 con 0x7ff804071d70 2024-03-13T07:33:51.644 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.642+0000 7ff7f2ffd640 1 -- 172.21.15.195:0/2430945880 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7ff7f80065d0 con 0x7ff804071d70 2024-03-13T07:33:51.644 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.642+0000 7ff7f2ffd640 10 client.4138.objecter ms_dispatch 0x7ff804000b90 osd_map(6..6 src has 1..6) v4 2024-03-13T07:33:51.644 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.642+0000 7ff7f2ffd640 3 client.4138.objecter handle_osd_map got epochs [6,6] > 0 2024-03-13T07:33:51.644 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.642+0000 7ff7f2ffd640 3 client.4138.objecter handle_osd_map decoding full epoch 6 2024-03-13T07:33:51.644 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.642+0000 7ff7f2ffd640 20 client.4138.objecter dump_active .. 0 homeless 2024-03-13T07:33:51.739 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.739+0000 7ff7f2ffd640 1 -- 172.21.15.195:0/2430945880 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff8041dbf70 con 0x7ff804071d70 2024-03-13T07:33:51.739 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.739+0000 7ff7f2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:51.739 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.739+0000 7ff7f2ffd640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:33:51.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.811+0000 7ff8088d3640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "1a719246-ae04-42b0-bd9c-fc318dc57ecb", "id": 4}] 2024-03-13T07:33:51.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.811+0000 7ff8088d3640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "1a719246-ae04-42b0-bd9c-fc318dc57ecb", "id": 4}] 2024-03-13T07:33:51.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.811+0000 7ff8088d3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:51.812 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.811+0000 7ff8088d3640 1 -- 172.21.15.195:0/2430945880 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1a719246-ae04-42b0-bd9c-fc318dc57ecb", "id": 4} v 0) v1 -- 0x7ff80404f000 con 0x7ff804071d70 2024-03-13T07:33:51.867 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.867+0000 7ff7f2ffd640 1 -- 172.21.15.195:0/2430945880 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1a719246-ae04-42b0-bd9c-fc318dc57ecb", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff7f8006820 con 0x7ff804071d70 2024-03-13T07:33:51.867 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.867+0000 7ff7f2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "1a719246-ae04-42b0-bd9c-fc318dc57ecb", "id": 4}] 2024-03-13T07:33:51.867 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.867+0000 7ff7f2ffd640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:33:51.868 INFO:teuthology.orchestra.run.smithi195.stdout:4 2024-03-13T07:33:51.869 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.868+0000 7ff8088d3640 10 librados: watch_flush enter 2024-03-13T07:33:51.869 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.868+0000 7ff8088d3640 10 librados: watch_flush exit 2024-03-13T07:33:51.869 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.868+0000 7ff8088d3640 20 client.4138.objecter shutdown clearing up homeless session... 2024-03-13T07:33:51.869 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.868+0000 7ff8088d3640 10 client.4138.objecter successfully canceled tick 2024-03-13T07:33:51.869 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.868+0000 7ff8088d3640 10 monclient: shutdown 2024-03-13T07:33:51.869 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.868+0000 7ff8088d3640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:51.869 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.868+0000 7ff8088d3640 1 -- 172.21.15.195:0/2430945880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff804071d70 msgr2=0x7ff8041db330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:51.870 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.868+0000 7ff8088d3640 1 --2- 172.21.15.195:0/2430945880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff804071d70 0x7ff8041db330 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800e9b0 tx=0x7ff7f800ee70 comp rx=0 tx=0).stop 2024-03-13T07:33:51.870 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.868+0000 7ff8088d3640 1 -- 172.21.15.195:0/2430945880 shutdown_connections 2024-03-13T07:33:51.870 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.868+0000 7ff8088d3640 1 --2- 172.21.15.195:0/2430945880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8040b1c00 0x7ff8041db870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.870 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.868+0000 7ff8088d3640 1 --2- 172.21.15.195:0/2430945880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff804071d70 0x7ff8041db330 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.870 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.868+0000 7ff8088d3640 1 --2- 172.21.15.195:0/2430945880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff804070bf0 0x7ff80413c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.870 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.868+0000 7ff8088d3640 1 -- 172.21.15.195:0/2430945880 >> 172.21.15.195:0/2430945880 conn(0x7ff8040b6050 msgr2=0x7ff8040b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:51.870 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.868+0000 7ff8088d3640 1 -- 172.21.15.195:0/2430945880 shutdown_connections 2024-03-13T07:33:51.870 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.868+0000 7ff8088d3640 1 -- 172.21.15.195:0/2430945880 wait complete. 2024-03-13T07:33:51.870 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.868+0000 7ff8088d3640 1 librados: shutdown 2024-03-13T07:33:51.879 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 9d9ee991-241b-4507-ba2f-e3b5e2844d1a 5 2024-03-13T07:33:51.978 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.978+0000 7f063a660640 10 monclient: get_monmap_and_config 2024-03-13T07:33:51.979 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.978+0000 7f063a660640 10 monclient: build_initial_monmap 2024-03-13T07:33:51.979 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.978+0000 7f063a660640 10 monclient: monmap: 2024-03-13T07:33:51.979 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 0 2024-03-13T07:33:51.979 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:51.979 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:51.979636+0000 2024-03-13T07:33:51.979 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:51.979636+0000 2024-03-13T07:33:51.979 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:51.979 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:51.979 INFO:teuthology.orchestra.run.smithi195.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:33:51.979 INFO:teuthology.orchestra.run.smithi195.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:33:51.979 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:33:51.980 INFO:teuthology.orchestra.run.smithi195.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:33:51.980 INFO:teuthology.orchestra.run.smithi195.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:33:51.980 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:51.982 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.979+0000 7f063a660640 1 Processor -- start 2024-03-13T07:33:51.982 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.979+0000 7f063a660640 1 -- start start 2024-03-13T07:33:51.982 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.979+0000 7f063a660640 10 monclient: init 2024-03-13T07:33:51.982 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.979+0000 7f063a660640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:51.982 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.979+0000 7f063a660640 10 monclient: _add_conns ranks=[0,3,2,1,4] 2024-03-13T07:33:51.982 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.979+0000 7f063a660640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f06340b5320 addr v1:172.21.15.132:6789/0 2024-03-13T07:33:51.982 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.979+0000 7f063a660640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f06340b0220 0x7f063413f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.982 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.979+0000 7f063a660640 10 monclient(hunting): picked mon.noname-a con 0x7f06340b0220 addr v2:172.21.15.132:3300/0 2024-03-13T07:33:51.982 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.979+0000 7f063a660640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f063413fb70 0x7f063414cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.982 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.980+0000 7f063a660640 10 monclient(hunting): picked mon.noname-b con 0x7f063413fb70 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:51.983 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.980+0000 7f063a660640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06340b1150 con 0x7f06340b5320 2024-03-13T07:33:51.983 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.980+0000 7f063a660640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.983 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.980+0000 7f063a660640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06340b0690 con 0x7f06340b0220 2024-03-13T07:33:51.983 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.980+0000 7f063a660640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.983 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.980+0000 7f063a660640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063413ffa0 con 0x7f063413fb70 2024-03-13T07:33:51.983 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.980+0000 7f063a660640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:51.983 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.980+0000 7f063a660640 10 monclient(hunting): authenticate will time out at 1246.318433s 2024-03-13T07:33:51.983 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.980+0000 7f0638bfd640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f063413fb70 0x7f063414cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.983 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.980+0000 7f06337fe640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f06340b0220 0x7f063413f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.983 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.981+0000 7f0638bfd640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f063413fb70 0x7f063414cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.195:48790/0 (socket says 172.21.15.195:48790) 2024-03-13T07:33:51.983 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.981+0000 7f0638bfd640 1 -- 172.21.15.195:0/3287040123 learned_addr learned my addr 172.21.15.195:0/3287040123 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-13T07:33:51.983 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.981+0000 7f0638bfd640 10 monclient(hunting): get_auth_request con 0x7f063413fb70 auth_method 0 2024-03-13T07:33:51.983 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.981+0000 7f0638bfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:51.984 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.981+0000 7f0638bfd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:51.984 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.981+0000 7f0638bfd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:51.984 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.981+0000 7f06337fe640 10 monclient(hunting): get_auth_request con 0x7f06340b0220 auth_method 0 2024-03-13T07:33:51.984 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.981+0000 7f06337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:51.984 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.981+0000 7f06337fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:51.984 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.981+0000 7f06337fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:51.984 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.981+0000 7f0638bfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:51.984 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.981+0000 7f0638bfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:51.984 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.981+0000 7f0638bfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:51.984 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.982+0000 7f06337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:51.984 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.982+0000 7f06337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:51.984 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.982+0000 7f06337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:51.985 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.983+0000 7f0638bfd640 10 monclient(hunting): handle_auth_done global_id 4130 payload 274 2024-03-13T07:33:51.985 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.983+0000 7f0638bfd640 1 -- 172.21.15.195:0/3287040123 >> v2:172.21.15.132:3300/0 conn(0x7f06340b0220 msgr2=0x7f063413f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:33:51.985 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.983+0000 7f0638bfd640 1 --2- 172.21.15.195:0/3287040123 >> v2:172.21.15.132:3300/0 conn(0x7f06340b0220 0x7f063413f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.985 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.983+0000 7f0638bfd640 1 -- 172.21.15.195:0/3287040123 >> v1:172.21.15.132:6789/0 conn(0x7f06340b5320 legacy=0x7f06340b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:51.985 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.983+0000 7f0638bfd640 10 monclient: _finish_hunting 0 2024-03-13T07:33:51.985 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.983+0000 7f0638bfd640 1 monclient: found mon.noname-b 2024-03-13T07:33:51.985 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.983+0000 7f0638bfd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:51.985 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.983+0000 7f0638bfd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T07:33:51.985 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.983+0000 7f0638bfd640 1 -- 172.21.15.195:0/3287040123 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06340b13c0 con 0x7f063413fb70 2024-03-13T07:33:51.986 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.986+0000 7f0638bfd640 1 --2- 172.21.15.195:0/3287040123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f063413fb70 0x7f063414cef0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f062c009e50 tx=0x7f062c02be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5341a2dea7896e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:51.989 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.989+0000 7f0632ffd640 1 -- 172.21.15.195:0/3287040123 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c02c9e0 con 0x7f063413fb70 2024-03-13T07:33:51.989 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.989+0000 7f0632ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:51.990 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.989+0000 7f0632ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:33:51.990 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.989+0000 7f0632ffd640 10 monclient: dump: 2024-03-13T07:33:51.990 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:51.990 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:51.990 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.990 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.990 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:51.990 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:51.990 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:51.990 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:51.990 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:51.990 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:51.990 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.989+0000 7f0632ffd640 10 monclient: _finish_auth 0 2024-03-13T07:33:51.990 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.989+0000 7f0632ffd640 10 monclient: _check_auth_tickets 2024-03-13T07:33:51.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.989+0000 7f0632ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:51.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.989+0000 7f0632ffd640 1 -- 172.21.15.195:0/3287040123 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c02cb80 con 0x7f063413fb70 2024-03-13T07:33:51.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.989+0000 7f0632ffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:51.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.989+0000 7f0632ffd640 1 -- 172.21.15.195:0/3287040123 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c0343f0 con 0x7f063413fb70 2024-03-13T07:33:51.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.989+0000 7f0632ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:51.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.989+0000 7f0632ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:33:51.991 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.989+0000 7f0632ffd640 10 monclient: dump: 2024-03-13T07:33:51.991 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:51.991 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:51.991 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.991 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:51.991 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:51.991 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:51.991 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:51.991 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:51.991 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:51.992 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:51.992 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.989+0000 7f063a660640 5 monclient: authenticate success, global_id 4130 2024-03-13T07:33:51.992 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.989+0000 7f063a660640 10 monclient: get_monmap_and_config success 2024-03-13T07:33:51.992 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.989+0000 7f063a660640 4 set_mon_vals no callback set 2024-03-13T07:33:51.995 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.990+0000 7f063a660640 10 monclient: shutdown 2024-03-13T07:33:51.995 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.990+0000 7f063a660640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:51.995 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.990+0000 7f063a660640 1 -- 172.21.15.195:0/3287040123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f063413fb70 msgr2=0x7f063414cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:51.995 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.990+0000 7f063a660640 1 --2- 172.21.15.195:0/3287040123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f063413fb70 0x7f063414cef0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f062c009e50 tx=0x7f062c02be40 comp rx=0 tx=0).stop 2024-03-13T07:33:51.995 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.990+0000 7f063a660640 1 -- 172.21.15.195:0/3287040123 shutdown_connections 2024-03-13T07:33:51.996 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.990+0000 7f063a660640 1 --2- 172.21.15.195:0/3287040123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f063413fb70 0x7f063414cef0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.996 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.990+0000 7f063a660640 1 --2- 172.21.15.195:0/3287040123 >> v2:172.21.15.132:3300/0 conn(0x7f06340b0220 0x7f063413f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:51.996 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.990+0000 7f063a660640 1 -- 172.21.15.195:0/3287040123 >> 172.21.15.195:0/3287040123 conn(0x7f063406d2b0 msgr2=0x7f063406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:51.996 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.990+0000 7f063a660640 1 -- 172.21.15.195:0/3287040123 shutdown_connections 2024-03-13T07:33:51.996 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.990+0000 7f063a660640 1 -- 172.21.15.195:0/3287040123 wait complete. 2024-03-13T07:33:51.996 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.991+0000 7f063a660640 10 monclient: build_initial_monmap 2024-03-13T07:33:51.996 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.991+0000 7f063a660640 10 monclient: monmap: 2024-03-13T07:33:51.996 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 0 2024-03-13T07:33:51.996 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:33:51.996 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 0.000000 2024-03-13T07:33:51.996 INFO:teuthology.orchestra.run.smithi195.stderr:created 0.000000 2024-03-13T07:33:51.996 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:51.996 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:51.996 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:33:51.996 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:33:51.997 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:33:51.997 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:51.997 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.991+0000 7f063a660640 1 librados: starting msgr at 2024-03-13T07:33:51.997 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.991+0000 7f063a660640 1 librados: starting objecter 2024-03-13T07:33:51.997 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.991+0000 7f063a660640 1 Processor -- start 2024-03-13T07:33:51.997 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.991+0000 7f063a660640 1 -- start start 2024-03-13T07:33:51.997 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.991+0000 7f063a660640 1 librados: setting wanted keys 2024-03-13T07:33:51.997 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.991+0000 7f063a660640 1 librados: calling monclient init 2024-03-13T07:33:51.997 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.991+0000 7f063a660640 10 monclient: init 2024-03-13T07:33:51.997 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f063a660640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:51.997 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f063a660640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T07:33:51.997 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f063a660640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06340b0220 0x7f06341429f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.997 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f063a660640 10 monclient(hunting): picked mon.noname-a con 0x7f06340b0220 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:33:51.997 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f063a660640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06340b5320 0x7f0634143630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.997 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f063a660640 10 monclient(hunting): picked mon.noname-c con 0x7f06340b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:51.998 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f063a660640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f063413fb70 0x7f0634145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:51.998 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f063a660640 10 monclient(hunting): picked mon.noname-b con 0x7f063413fb70 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:33:51.998 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f063a660640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.998 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f063a660640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06340afd00 con 0x7f06340b0220 2024-03-13T07:33:51.998 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f063a660640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.998 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f063a660640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06340afa20 con 0x7f063413fb70 2024-03-13T07:33:51.998 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f063a660640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:51.998 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f063a660640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06340b4800 con 0x7f06340b5320 2024-03-13T07:33:51.998 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f063a660640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:51.998 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f063a660640 10 monclient(hunting): authenticate will time out at 1246.330868s 2024-03-13T07:33:51.998 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f06337fe640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06340b5320 0x7f0634143630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.998 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.992+0000 7f0638bfd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f063413fb70 0x7f0634145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.998 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f06337fe640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06340b5320 0x7f0634143630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.195:48798/0 (socket says 172.21.15.195:48798) 2024-03-13T07:33:51.998 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f06337fe640 1 -- 172.21.15.195:0/4071109541 learned_addr learned my addr 172.21.15.195:0/4071109541 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-13T07:33:51.999 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f06337fe640 10 monclient(hunting): get_auth_request con 0x7f06340b5320 auth_method 0 2024-03-13T07:33:51.999 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f06337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:51.999 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f06337fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:51.999 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f06337fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:51.999 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f0633fff640 1 --2- 172.21.15.195:0/4071109541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06340b0220 0x7f06341429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:51.999 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f0638bfd640 10 monclient(hunting): get_auth_request con 0x7f063413fb70 auth_method 0 2024-03-13T07:33:51.999 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f0638bfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:51.999 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f0638bfd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:51.999 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f0638bfd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:51.999 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f0633fff640 10 monclient(hunting): get_auth_request con 0x7f06340b0220 auth_method 0 2024-03-13T07:33:51.999 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f0633fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:51.999 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f0633fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:51.999 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f0633fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:51.999 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f06337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:51.999 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f06337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:51.999 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f06337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:52.000 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f0638bfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:52.000 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f0638bfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:52.000 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.993+0000 7f0638bfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:52.000 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.994+0000 7f0633fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:52.000 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.994+0000 7f0633fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:52.000 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.994+0000 7f0633fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:52.001 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.996+0000 7f0638bfd640 10 monclient(hunting): handle_auth_done global_id 4141 payload 824 2024-03-13T07:33:52.001 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.996+0000 7f0638bfd640 1 -- 172.21.15.195:0/4071109541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06340b5320 msgr2=0x7f0634143630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:52.001 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.996+0000 7f0638bfd640 1 --2- 172.21.15.195:0/4071109541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06340b5320 0x7f0634143630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.001 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.996+0000 7f0638bfd640 1 -- 172.21.15.195:0/4071109541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06340b0220 msgr2=0x7f06341429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:52.001 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.996+0000 7f0638bfd640 1 --2- 172.21.15.195:0/4071109541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06340b0220 0x7f06341429f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.001 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.996+0000 7f0638bfd640 10 monclient: _finish_hunting 0 2024-03-13T07:33:52.001 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.996+0000 7f0638bfd640 1 monclient: found mon.noname-b 2024-03-13T07:33:52.001 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.996+0000 7f0638bfd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:52.001 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.996+0000 7f0638bfd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:33:52.001 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.997+0000 7f0638bfd640 1 -- 172.21.15.195:0/4071109541 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0634145c30 con 0x7f063413fb70 2024-03-13T07:33:52.001 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.999+0000 7f0638bfd640 1 --2- 172.21.15.195:0/4071109541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f063413fb70 0x7f0634145180 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f062c032000 tx=0x7f062c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:52.001 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.999+0000 7f0630ff9640 10 client.?.objecter ms_handle_connect 0x7f063413fb70 2024-03-13T07:33:52.002 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:51.999+0000 7f0630ff9640 10 client.?.objecter resend_mon_ops 2024-03-13T07:33:52.003 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f0630ff9640 1 -- 172.21.15.195:0/4071109541 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c043020 con 0x7f063413fb70 2024-03-13T07:33:52.003 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f0630ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:52.003 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f0630ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:33:52.003 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f0630ff9640 10 monclient: dump: 2024-03-13T07:33:52.003 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:52.003 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:52.003 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.003 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.003 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:52.004 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:52.004 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:52.004 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:52.004 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:52.004 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:52.004 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f0630ff9640 10 monclient: _finish_auth 0 2024-03-13T07:33:52.004 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f0630ff9640 10 monclient: _check_auth_tickets 2024-03-13T07:33:52.004 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f0630ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:52.004 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f0630ff9640 1 -- 172.21.15.195:0/4071109541 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c03e450 con 0x7f063413fb70 2024-03-13T07:33:52.004 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f0630ff9640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:52.004 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f0630ff9640 1 -- 172.21.15.195:0/4071109541 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c009510 con 0x7f063413fb70 2024-03-13T07:33:52.004 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f063a660640 5 monclient: authenticate success, global_id 4141 2024-03-13T07:33:52.004 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f063a660640 10 monclient: _renew_subs 2024-03-13T07:33:52.004 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f063a660640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:52.005 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f063a660640 1 -- 172.21.15.195:0/4071109541 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0634143b70 con 0x7f063413fb70 2024-03-13T07:33:52.005 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f06327fc640 4 set_mon_vals no callback set 2024-03-13T07:33:52.005 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f0630ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:52.005 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f0630ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:33:52.005 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.002+0000 7f0630ff9640 10 monclient: dump: 2024-03-13T07:33:52.005 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:52.005 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:52.005 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.005 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.005 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:52.005 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:52.005 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:52.005 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:52.005 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:52.005 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:52.006 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.003+0000 7f063a660640 10 client.4141.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:33:52.006 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.003+0000 7f063a660640 10 monclient: _renew_subs 2024-03-13T07:33:52.006 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.003+0000 7f063a660640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:52.006 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.003+0000 7f063a660640 1 -- 172.21.15.195:0/4071109541 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0634146150 con 0x7f063413fb70 2024-03-13T07:33:52.006 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.003+0000 7f063a660640 1 librados: init done 2024-03-13T07:33:52.006 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.003+0000 7f063a660640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:52.006 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.003+0000 7f063a660640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:52.006 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.003+0000 7f063a660640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:52.006 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.003+0000 7f063a660640 1 -- 172.21.15.195:0/4071109541 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06340b7e70 con 0x7f063413fb70 2024-03-13T07:33:52.106 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.105+0000 7f0630ff9640 1 -- 172.21.15.195:0/4071109541 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f062c046020 con 0x7f063413fb70 2024-03-13T07:33:52.106 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.105+0000 7f0630ff9640 1 -- 172.21.15.195:0/4071109541 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f062c0519b0 con 0x7f063413fb70 2024-03-13T07:33:52.106 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.105+0000 7f0630ff9640 10 client.4141.objecter ms_dispatch 0x7f0634000b90 osd_map(7..7 src has 1..7) v4 2024-03-13T07:33:52.106 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.105+0000 7f0630ff9640 3 client.4141.objecter handle_osd_map got epochs [7,7] > 0 2024-03-13T07:33:52.106 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.105+0000 7f0630ff9640 3 client.4141.objecter handle_osd_map decoding full epoch 7 2024-03-13T07:33:52.106 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.105+0000 7f0630ff9640 20 client.4141.objecter dump_active .. 0 homeless 2024-03-13T07:33:52.106 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.105+0000 7f0630ff9640 1 -- 172.21.15.195:0/4071109541 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f06340b7e70 con 0x7f063413fb70 2024-03-13T07:33:52.107 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.105+0000 7f0630ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:52.107 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.105+0000 7f0630ff9640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:33:52.179 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.178+0000 7f063a660640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "9d9ee991-241b-4507-ba2f-e3b5e2844d1a", "id": 5}] 2024-03-13T07:33:52.179 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.178+0000 7f063a660640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "9d9ee991-241b-4507-ba2f-e3b5e2844d1a", "id": 5}] 2024-03-13T07:33:52.179 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.178+0000 7f063a660640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:52.179 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.178+0000 7f063a660640 1 -- 172.21.15.195:0/4071109541 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9d9ee991-241b-4507-ba2f-e3b5e2844d1a", "id": 5} v 0) v1 -- 0x7f063404f000 con 0x7f063413fb70 2024-03-13T07:33:52.239 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.239+0000 7f0630ff9640 1 -- 172.21.15.195:0/4071109541 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9d9ee991-241b-4507-ba2f-e3b5e2844d1a", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f062c04d030 con 0x7f063413fb70 2024-03-13T07:33:52.239 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.239+0000 7f0630ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "9d9ee991-241b-4507-ba2f-e3b5e2844d1a", "id": 5}] 2024-03-13T07:33:52.240 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.239+0000 7f0630ff9640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:33:52.240 INFO:teuthology.orchestra.run.smithi195.stdout:5 2024-03-13T07:33:52.241 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.240+0000 7f063a660640 10 librados: watch_flush enter 2024-03-13T07:33:52.241 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.240+0000 7f063a660640 10 librados: watch_flush exit 2024-03-13T07:33:52.241 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.240+0000 7f063a660640 20 client.4141.objecter shutdown clearing up homeless session... 2024-03-13T07:33:52.242 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.240+0000 7f063a660640 10 client.4141.objecter successfully canceled tick 2024-03-13T07:33:52.242 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.240+0000 7f063a660640 10 monclient: shutdown 2024-03-13T07:33:52.242 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.240+0000 7f063a660640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:52.242 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.240+0000 7f063a660640 1 -- 172.21.15.195:0/4071109541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f063413fb70 msgr2=0x7f0634145180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:52.242 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.240+0000 7f063a660640 1 --2- 172.21.15.195:0/4071109541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f063413fb70 0x7f0634145180 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f062c032000 tx=0x7f062c004a40 comp rx=0 tx=0).stop 2024-03-13T07:33:52.242 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.240+0000 7f063a660640 1 -- 172.21.15.195:0/4071109541 shutdown_connections 2024-03-13T07:33:52.242 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.240+0000 7f063a660640 1 --2- 172.21.15.195:0/4071109541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f063413fb70 0x7f0634145180 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.242 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.240+0000 7f063a660640 1 --2- 172.21.15.195:0/4071109541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06340b5320 0x7f0634143630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.242 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.240+0000 7f063a660640 1 --2- 172.21.15.195:0/4071109541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06340b0220 0x7f06341429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.242 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.240+0000 7f063a660640 1 -- 172.21.15.195:0/4071109541 >> 172.21.15.195:0/4071109541 conn(0x7f063406d2b0 msgr2=0x7f0634074530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:52.242 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.240+0000 7f063a660640 1 -- 172.21.15.195:0/4071109541 shutdown_connections 2024-03-13T07:33:52.242 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.240+0000 7f063a660640 1 -- 172.21.15.195:0/4071109541 wait complete. 2024-03-13T07:33:52.243 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.240+0000 7f063a660640 1 librados: shutdown 2024-03-13T07:33:52.253 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new af8b34fd-927b-4860-971b-9fddf699a0dc 6 2024-03-13T07:33:52.354 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.354+0000 7f958afef640 10 monclient: get_monmap_and_config 2024-03-13T07:33:52.354 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.354+0000 7f958afef640 10 monclient: build_initial_monmap 2024-03-13T07:33:52.354 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.354+0000 7f958afef640 10 monclient: monmap: 2024-03-13T07:33:52.355 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 0 2024-03-13T07:33:52.355 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:52.355 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:52.355597+0000 2024-03-13T07:33:52.355 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:52.355597+0000 2024-03-13T07:33:52.355 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:52.355 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:52.355 INFO:teuthology.orchestra.run.smithi195.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:33:52.355 INFO:teuthology.orchestra.run.smithi195.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:33:52.355 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:33:52.355 INFO:teuthology.orchestra.run.smithi195.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:33:52.355 INFO:teuthology.orchestra.run.smithi195.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:33:52.355 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:52.358 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 1 Processor -- start 2024-03-13T07:33:52.358 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 1 -- start start 2024-03-13T07:33:52.358 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 10 monclient: init 2024-03-13T07:33:52.358 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:52.358 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 10 monclient: _add_conns ranks=[3,2,0,1,4] 2024-03-13T07:33:52.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f95840b5320 0x7f95840b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:52.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 10 monclient(hunting): picked mon.noname-a con 0x7f95840b5320 addr v2:172.21.15.132:3300/0 2024-03-13T07:33:52.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95840b1ac0 0x7f95840b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:52.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 10 monclient(hunting): picked mon.noname-b con 0x7f95840b1ac0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:52.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f958413f630 addr v1:172.21.15.132:6789/0 2024-03-13T07:33:52.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95840b1150 con 0x7f958413f630 2024-03-13T07:33:52.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:52.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95840b5700 con 0x7f95840b5320 2024-03-13T07:33:52.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:52.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95840b13c0 con 0x7f95840b1ac0 2024-03-13T07:33:52.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:52.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.355+0000 7f958afef640 10 monclient(hunting): authenticate will time out at 1246.694356s 2024-03-13T07:33:52.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.356+0000 7f9583fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95840b1ac0 0x7f95840b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:52.359 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.356+0000 7f9588d8b640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f95840b5320 0x7f95840b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:52.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.356+0000 7f9583fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95840b1ac0 0x7f95840b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.195:48806/0 (socket says 172.21.15.195:48806) 2024-03-13T07:33:52.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.356+0000 7f9583fff640 1 -- 172.21.15.195:0/3524801452 learned_addr learned my addr 172.21.15.195:0/3524801452 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-13T07:33:52.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.356+0000 7f9583fff640 10 monclient(hunting): get_auth_request con 0x7f95840b1ac0 auth_method 0 2024-03-13T07:33:52.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.356+0000 7f9583fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:52.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.356+0000 7f9583fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:52.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.356+0000 7f9583fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:52.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.357+0000 7f9588d8b640 10 monclient(hunting): get_auth_request con 0x7f95840b5320 auth_method 0 2024-03-13T07:33:52.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.357+0000 7f9588d8b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:52.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.357+0000 7f9588d8b640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:52.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.357+0000 7f9588d8b640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:52.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.357+0000 7f9583fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:52.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.357+0000 7f9583fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:52.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.357+0000 7f9583fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:52.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.358+0000 7f9588d8b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:52.361 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.358+0000 7f9588d8b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:52.361 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.358+0000 7f9588d8b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:52.361 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:52.357+0000 7fb6b5844040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-13T07:33:52.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.359+0000 7f9583fff640 10 monclient(hunting): handle_auth_done global_id 4136 payload 274 2024-03-13T07:33:52.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.359+0000 7f9583fff640 1 -- 172.21.15.195:0/3524801452 >> v2:172.21.15.132:3300/0 conn(0x7f95840b5320 msgr2=0x7f95840b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:33:52.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.359+0000 7f9583fff640 1 --2- 172.21.15.195:0/3524801452 >> v2:172.21.15.132:3300/0 conn(0x7f95840b5320 0x7f95840b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.359+0000 7f9583fff640 1 -- 172.21.15.195:0/3524801452 >> v1:172.21.15.132:6789/0 conn(0x7f958413f630 legacy=0x7f958413fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:52.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.359+0000 7f9583fff640 10 monclient: _finish_hunting 0 2024-03-13T07:33:52.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.359+0000 7f9583fff640 1 monclient: found mon.noname-b 2024-03-13T07:33:52.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.359+0000 7f9583fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:52.363 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.359+0000 7f9583fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T07:33:52.363 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.359+0000 7f9583fff640 1 -- 172.21.15.195:0/3524801452 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95840b16a0 con 0x7f95840b1ac0 2024-03-13T07:33:52.363 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.361+0000 7f9583fff640 1 --2- 172.21.15.195:0/3524801452 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95840b1ac0 0x7f95840b1f00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9570009870 tx=0x7f957002b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbbd04b0fe0cbbed server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:52.365 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.365+0000 7f95837fe640 1 -- 172.21.15.195:0/3524801452 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95700317c0 con 0x7f95840b1ac0 2024-03-13T07:33:52.365 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.365+0000 7f95837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:52.366 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.365+0000 7f95837fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:33:52.366 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.365+0000 7f95837fe640 10 monclient: dump: 2024-03-13T07:33:52.366 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:52.366 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:52.366 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.366 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.366 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:52.366 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:52.366 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:52.366 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:52.366 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:52.366 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:52.366 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.365+0000 7f95837fe640 10 monclient: _finish_auth 0 2024-03-13T07:33:52.366 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.365+0000 7f95837fe640 10 monclient: _check_auth_tickets 2024-03-13T07:33:52.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.365+0000 7f95837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:52.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.365+0000 7f95837fe640 1 -- 172.21.15.195:0/3524801452 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9570037070 con 0x7f95840b1ac0 2024-03-13T07:33:52.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.365+0000 7f95837fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:52.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.365+0000 7f95837fe640 1 -- 172.21.15.195:0/3524801452 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570031b80 con 0x7f95840b1ac0 2024-03-13T07:33:52.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.365+0000 7f95837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:52.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.365+0000 7f95837fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:33:52.367 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.365+0000 7f95837fe640 10 monclient: dump: 2024-03-13T07:33:52.367 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:52.367 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:52.367 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.367 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.367 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:52.367 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:52.367 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:52.367 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:52.368 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:52.368 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:52.368 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.365+0000 7f958afef640 5 monclient: authenticate success, global_id 4136 2024-03-13T07:33:52.368 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.365+0000 7f958afef640 10 monclient: get_monmap_and_config success 2024-03-13T07:33:52.368 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.365+0000 7f958afef640 4 set_mon_vals no callback set 2024-03-13T07:33:52.371 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.366+0000 7f958afef640 10 monclient: shutdown 2024-03-13T07:33:52.372 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.366+0000 7f958afef640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:52.372 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.366+0000 7f958afef640 1 -- 172.21.15.195:0/3524801452 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95840b1ac0 msgr2=0x7f95840b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:52.372 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.366+0000 7f958afef640 1 --2- 172.21.15.195:0/3524801452 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95840b1ac0 0x7f95840b1f00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9570009870 tx=0x7f957002b860 comp rx=0 tx=0).stop 2024-03-13T07:33:52.372 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.366+0000 7f958afef640 1 -- 172.21.15.195:0/3524801452 shutdown_connections 2024-03-13T07:33:52.372 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.366+0000 7f958afef640 1 --2- 172.21.15.195:0/3524801452 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95840b1ac0 0x7f95840b1f00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.372 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.366+0000 7f958afef640 1 --2- 172.21.15.195:0/3524801452 >> v2:172.21.15.132:3300/0 conn(0x7f95840b5320 0x7f95840b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.372 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.366+0000 7f958afef640 1 -- 172.21.15.195:0/3524801452 >> 172.21.15.195:0/3524801452 conn(0x7f958406d2b0 msgr2=0x7f958406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:52.372 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.366+0000 7f958afef640 1 -- 172.21.15.195:0/3524801452 shutdown_connections 2024-03-13T07:33:52.372 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.366+0000 7f958afef640 1 -- 172.21.15.195:0/3524801452 wait complete. 2024-03-13T07:33:52.372 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.366+0000 7f958afef640 10 monclient: build_initial_monmap 2024-03-13T07:33:52.372 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.366+0000 7f958afef640 10 monclient: monmap: 2024-03-13T07:33:52.372 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 0 2024-03-13T07:33:52.373 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:33:52.373 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 0.000000 2024-03-13T07:33:52.373 INFO:teuthology.orchestra.run.smithi195.stderr:created 0.000000 2024-03-13T07:33:52.373 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:52.373 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:52.373 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:33:52.373 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:33:52.373 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:33:52.373 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:52.374 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 1 librados: starting msgr at 2024-03-13T07:33:52.374 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 1 librados: starting objecter 2024-03-13T07:33:52.374 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 1 Processor -- start 2024-03-13T07:33:52.374 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 1 -- start start 2024-03-13T07:33:52.374 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 1 librados: setting wanted keys 2024-03-13T07:33:52.374 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 1 librados: calling monclient init 2024-03-13T07:33:52.374 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 10 monclient: init 2024-03-13T07:33:52.374 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:52.374 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T07:33:52.374 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95840b1ac0 0x7f95841425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:52.374 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 10 monclient(hunting): picked mon.noname-a con 0x7f95840b1ac0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:33:52.375 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95840b5320 0x7f95841383c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:52.375 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 10 monclient(hunting): picked mon.noname-c con 0x7f95840b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:52.375 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f958413f630 0x7f95840aa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:52.375 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 10 monclient(hunting): picked mon.noname-b con 0x7f958413f630 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:33:52.375 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:52.375 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95840b4980 con 0x7f95840b1ac0 2024-03-13T07:33:52.375 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:52.375 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95840b4800 con 0x7f958413f630 2024-03-13T07:33:52.375 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:52.375 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958414cef0 con 0x7f95840b5320 2024-03-13T07:33:52.375 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:52.375 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.367+0000 7f958afef640 10 monclient(hunting): authenticate will time out at 1246.706369s 2024-03-13T07:33:52.375 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.368+0000 7f9583fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95840b5320 0x7f95841383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:52.376 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.368+0000 7f958958c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f958413f630 0x7f95840aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:52.376 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.368+0000 7f9588d8b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95840b1ac0 0x7f95841425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:52.376 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.368+0000 7f9583fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95840b5320 0x7f95841383c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.195:48814/0 (socket says 172.21.15.195:48814) 2024-03-13T07:33:52.376 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.368+0000 7f9583fff640 1 -- 172.21.15.195:0/1074434179 learned_addr learned my addr 172.21.15.195:0/1074434179 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-13T07:33:52.376 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.368+0000 7f9583fff640 10 monclient(hunting): get_auth_request con 0x7f95840b5320 auth_method 0 2024-03-13T07:33:52.376 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.368+0000 7f9583fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:52.376 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.368+0000 7f9583fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:52.376 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.368+0000 7f9583fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:52.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.368+0000 7f958958c640 10 monclient(hunting): get_auth_request con 0x7f958413f630 auth_method 0 2024-03-13T07:33:52.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.368+0000 7f958958c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:52.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.368+0000 7f958958c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:52.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.368+0000 7f958958c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:52.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.369+0000 7f9588d8b640 10 monclient(hunting): get_auth_request con 0x7f95840b1ac0 auth_method 0 2024-03-13T07:33:52.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.369+0000 7f9588d8b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:52.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.369+0000 7f9588d8b640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:52.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.369+0000 7f9588d8b640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:52.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.369+0000 7f9583fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:52.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.369+0000 7f9583fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:52.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.369+0000 7f9583fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:52.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.369+0000 7f958958c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:52.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.369+0000 7f958958c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:52.377 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.369+0000 7f958958c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:52.378 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.369+0000 7f9588d8b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:52.378 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.369+0000 7f9588d8b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:52.378 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.369+0000 7f9588d8b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:52.380 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.372+0000 7f958958c640 10 monclient(hunting): handle_auth_done global_id 4144 payload 824 2024-03-13T07:33:52.380 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.372+0000 7f958958c640 1 -- 172.21.15.195:0/1074434179 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95840b5320 msgr2=0x7f95841383c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:52.380 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.372+0000 7f958958c640 1 --2- 172.21.15.195:0/1074434179 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95840b5320 0x7f95841383c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.380 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.372+0000 7f958958c640 1 -- 172.21.15.195:0/1074434179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95840b1ac0 msgr2=0x7f95841425e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:52.380 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.372+0000 7f958958c640 1 --2- 172.21.15.195:0/1074434179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95840b1ac0 0x7f95841425e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.380 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.372+0000 7f958958c640 10 monclient: _finish_hunting 0 2024-03-13T07:33:52.380 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.372+0000 7f958958c640 1 monclient: found mon.noname-b 2024-03-13T07:33:52.380 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.372+0000 7f958958c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:52.380 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.372+0000 7f958958c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:33:52.380 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.372+0000 7f958958c640 1 -- 172.21.15.195:0/1074434179 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9584138900 con 0x7f958413f630 2024-03-13T07:33:52.381 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.374+0000 7f958958c640 1 --2- 172.21.15.195:0/1074434179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f958413f630 0x7f95840aa940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f957c00c700 tx=0x7f957c00cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:52.381 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.375+0000 7f95817fa640 10 client.?.objecter ms_handle_connect 0x7f958413f630 2024-03-13T07:33:52.381 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.375+0000 7f95817fa640 10 client.?.objecter resend_mon_ops 2024-03-13T07:33:52.381 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f95817fa640 1 -- 172.21.15.195:0/1074434179 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c003b70 con 0x7f958413f630 2024-03-13T07:33:52.381 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f95817fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:52.381 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f95817fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:33:52.381 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f95817fa640 10 monclient: dump: 2024-03-13T07:33:52.381 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:52.381 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:52.381 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.381 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.381 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:52.381 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:52.381 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:52.381 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:52.382 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:52.382 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:52.382 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f95817fa640 10 monclient: _finish_auth 0 2024-03-13T07:33:52.382 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f95817fa640 10 monclient: _check_auth_tickets 2024-03-13T07:33:52.382 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f95817fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:52.382 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f95817fa640 1 -- 172.21.15.195:0/1074434179 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957c003d10 con 0x7f958413f630 2024-03-13T07:33:52.382 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f958afef640 5 monclient: authenticate success, global_id 4144 2024-03-13T07:33:52.382 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f958afef640 10 monclient: _renew_subs 2024-03-13T07:33:52.382 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f958afef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:52.382 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f958afef640 1 -- 172.21.15.195:0/1074434179 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9584138b90 con 0x7f958413f630 2024-03-13T07:33:52.382 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f95817fa640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:52.382 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f95817fa640 1 -- 172.21.15.195:0/1074434179 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c005310 con 0x7f958413f630 2024-03-13T07:33:52.383 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f95817fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:52.383 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f95817fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:33:52.383 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f95817fa640 10 monclient: dump: 2024-03-13T07:33:52.383 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:52.383 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:52.383 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.383 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.383 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:52.383 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:52.383 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:52.383 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:52.383 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:52.383 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:52.384 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f9582ffd640 4 set_mon_vals no callback set 2024-03-13T07:33:52.384 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f958afef640 10 client.4144.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:33:52.384 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f958afef640 10 monclient: _renew_subs 2024-03-13T07:33:52.384 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f958afef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:52.384 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f958afef640 1 -- 172.21.15.195:0/1074434179 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95840ab060 con 0x7f958413f630 2024-03-13T07:33:52.384 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.378+0000 7f958afef640 1 librados: init done 2024-03-13T07:33:52.384 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.379+0000 7f958afef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:52.384 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.379+0000 7f958afef640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:52.384 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.379+0000 7f958afef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:52.384 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.379+0000 7f958afef640 1 -- 172.21.15.195:0/1074434179 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958404f000 con 0x7f958413f630 2024-03-13T07:33:52.385 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.381+0000 7f95817fa640 1 -- 172.21.15.195:0/1074434179 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f957c0054b0 con 0x7f958413f630 2024-03-13T07:33:52.385 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.381+0000 7f95817fa640 1 -- 172.21.15.195:0/1074434179 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f957c025a20 con 0x7f958413f630 2024-03-13T07:33:52.385 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.381+0000 7f95817fa640 10 client.4144.objecter ms_dispatch 0x7f9584000b90 osd_map(8..8 src has 1..8) v4 2024-03-13T07:33:52.385 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.381+0000 7f95817fa640 3 client.4144.objecter handle_osd_map got epochs [8,8] > 0 2024-03-13T07:33:52.385 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.381+0000 7f95817fa640 3 client.4144.objecter handle_osd_map decoding full epoch 8 2024-03-13T07:33:52.385 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.381+0000 7f95817fa640 20 client.4144.objecter dump_active .. 0 homeless 2024-03-13T07:33:52.410 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:52.409+0000 7fd30663e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-13T07:33:52.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.479+0000 7f95817fa640 1 -- 172.21.15.195:0/1074434179 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f957c0057a0 con 0x7f958413f630 2024-03-13T07:33:52.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.479+0000 7f95817fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:52.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.479+0000 7f95817fa640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:33:52.552 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.552+0000 7f958afef640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "af8b34fd-927b-4860-971b-9fddf699a0dc", "id": 6}] 2024-03-13T07:33:52.552 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.552+0000 7f958afef640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "af8b34fd-927b-4860-971b-9fddf699a0dc", "id": 6}] 2024-03-13T07:33:52.552 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.552+0000 7f958afef640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:33:52.552 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.552+0000 7f958afef640 1 -- 172.21.15.195:0/1074434179 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "af8b34fd-927b-4860-971b-9fddf699a0dc", "id": 6} v 0) v1 -- 0x7f95840ab640 con 0x7f958413f630 2024-03-13T07:33:52.611 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.611+0000 7f95817fa640 1 -- 172.21.15.195:0/1074434179 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "af8b34fd-927b-4860-971b-9fddf699a0dc", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f957c025c20 con 0x7f958413f630 2024-03-13T07:33:52.612 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.611+0000 7f95817fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "af8b34fd-927b-4860-971b-9fddf699a0dc", "id": 6}] 2024-03-13T07:33:52.612 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.611+0000 7f95817fa640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:33:52.612 INFO:teuthology.orchestra.run.smithi195.stdout:6 2024-03-13T07:33:52.613 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.612+0000 7f958afef640 10 librados: watch_flush enter 2024-03-13T07:33:52.613 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.612+0000 7f958afef640 10 librados: watch_flush exit 2024-03-13T07:33:52.613 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.612+0000 7f958afef640 20 client.4144.objecter shutdown clearing up homeless session... 2024-03-13T07:33:52.613 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.612+0000 7f958afef640 10 client.4144.objecter successfully canceled tick 2024-03-13T07:33:52.614 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.612+0000 7f958afef640 10 monclient: shutdown 2024-03-13T07:33:52.614 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.612+0000 7f958afef640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:52.614 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.612+0000 7f958afef640 1 -- 172.21.15.195:0/1074434179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f958413f630 msgr2=0x7f95840aa940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:52.614 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.612+0000 7f958afef640 1 --2- 172.21.15.195:0/1074434179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f958413f630 0x7f95840aa940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f957c00c700 tx=0x7f957c00cbc0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.614 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.612+0000 7f958afef640 1 -- 172.21.15.195:0/1074434179 shutdown_connections 2024-03-13T07:33:52.614 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.612+0000 7f958afef640 1 --2- 172.21.15.195:0/1074434179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f958413f630 0x7f95840aa940 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.614 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.612+0000 7f958afef640 1 --2- 172.21.15.195:0/1074434179 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95840b5320 0x7f95841383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.614 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.612+0000 7f958afef640 1 --2- 172.21.15.195:0/1074434179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95840b1ac0 0x7f95841425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.614 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.612+0000 7f958afef640 1 -- 172.21.15.195:0/1074434179 >> 172.21.15.195:0/1074434179 conn(0x7f958406d2b0 msgr2=0x7f9584074530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:52.614 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.613+0000 7f958afef640 1 -- 172.21.15.195:0/1074434179 shutdown_connections 2024-03-13T07:33:52.614 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.613+0000 7f958afef640 1 -- 172.21.15.195:0/1074434179 wait complete. 2024-03-13T07:33:52.614 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.613+0000 7f958afef640 1 librados: shutdown 2024-03-13T07:33:52.623 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 19a39645-8664-463b-bae3-d7aa1026e5f9 7 2024-03-13T07:33:52.638 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:52.638+0000 7fb6b5844040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-13T07:33:52.689 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:52.689+0000 7fb6b5844040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-13T07:33:52.700 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:52.700+0000 7fd30663e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-13T07:33:52.722 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.722+0000 7f1e12e53640 10 monclient: get_monmap_and_config 2024-03-13T07:33:52.723 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.722+0000 7f1e12e53640 10 monclient: build_initial_monmap 2024-03-13T07:33:52.723 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.722+0000 7f1e12e53640 10 monclient: monmap: 2024-03-13T07:33:52.723 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 0 2024-03-13T07:33:52.723 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:52.723 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:52.723387+0000 2024-03-13T07:33:52.723 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:52.723387+0000 2024-03-13T07:33:52.723 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:52.723 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:52.723 INFO:teuthology.orchestra.run.smithi195.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:33:52.723 INFO:teuthology.orchestra.run.smithi195.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:33:52.724 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:33:52.724 INFO:teuthology.orchestra.run.smithi195.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:33:52.724 INFO:teuthology.orchestra.run.smithi195.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:33:52.724 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:52.724 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.723+0000 7f1e12e53640 1 Processor -- start 2024-03-13T07:33:52.724 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.723+0000 7f1e12e53640 1 -- start start 2024-03-13T07:33:52.724 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.723+0000 7f1e12e53640 10 monclient: init 2024-03-13T07:33:52.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.723+0000 7f1e12e53640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:52.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.723+0000 7f1e12e53640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2024-03-13T07:33:52.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.723+0000 7f1e12e53640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0c0b5320 0x7f1e0c0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:52.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.723+0000 7f1e12e53640 10 monclient(hunting): picked mon.noname-b con 0x7f1e0c0b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:52.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.723+0000 7f1e12e53640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1e0c0b1ac0 addr v1:172.21.15.132:6789/0 2024-03-13T07:33:52.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.723+0000 7f1e12e53640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1e0c13fd40 addr v1:172.21.15.195:6789/0 2024-03-13T07:33:52.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.723+0000 7f1e12e53640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c0b1150 con 0x7f1e0c0b1ac0 2024-03-13T07:33:52.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.723+0000 7f1e12e53640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c0b13c0 con 0x7f1e0c13fd40 2024-03-13T07:33:52.726 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.723+0000 7f1e12e53640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:52.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.723+0000 7f1e12e53640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0b16a0 con 0x7f1e0c0b5320 2024-03-13T07:33:52.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.723+0000 7f1e12e53640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:52.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.723+0000 7f1e12e53640 10 monclient(hunting): authenticate will time out at 1247.062124s 2024-03-13T07:33:52.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.724+0000 7f1e10bef640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0c0b5320 0x7f1e0c0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:52.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.724+0000 7f1e113f0640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f1e0c13fd40 0x7f1e0c14cef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.195:56012/0 (socket says 172.21.15.195:56012) 2024-03-13T07:33:52.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.724+0000 7f1e113f0640 1 -- 172.21.15.195:0/267460572 learned_addr learned my addr 172.21.15.195:0/267460572 (peer_addr_for_me v1:172.21.15.195:0/0) 2024-03-13T07:33:52.727 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.724+0000 7f1e10bef640 10 monclient(hunting): get_auth_request con 0x7f1e0c0b5320 auth_method 0 2024-03-13T07:33:52.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.724+0000 7f1e10bef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:52.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.724+0000 7f1e10bef640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:52.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.724+0000 7f1e10bef640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:52.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.725+0000 7f1e10bef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:52.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.725+0000 7f1e10bef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:52.728 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.725+0000 7f1e10bef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:52.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.726+0000 7f1e0b7fe640 1 -- 172.21.15.195:0/267460572 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386334432 0 0) 0x7f1e0c0b13c0 con 0x7f1e0c13fd40 2024-03-13T07:33:52.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.726+0000 7f1e0b7fe640 10 monclient(hunting): _negotiate 2024-03-13T07:33:52.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.726+0000 7f1e0b7fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:52.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.726+0000 7f1e0b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:52.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.726+0000 7f1e0b7fe640 10 monclient(hunting): my global_id is 4147 2024-03-13T07:33:52.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.726+0000 7f1e0b7fe640 1 -- 172.21.15.195:0/267460572 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dfc003520 con 0x7f1e0c13fd40 2024-03-13T07:33:52.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.727+0000 7f1e10bef640 10 monclient(hunting): handle_auth_done global_id 4142 payload 274 2024-03-13T07:33:52.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.727+0000 7f1e10bef640 1 -- 172.21.15.195:0/267460572 >> v1:172.21.15.195:6789/0 conn(0x7f1e0c13fd40 legacy=0x7f1e0c14cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:52.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.727+0000 7f1e10bef640 1 -- 172.21.15.195:0/267460572 >> v1:172.21.15.132:6789/0 conn(0x7f1e0c0b1ac0 legacy=0x7f1e0c13f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:52.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.727+0000 7f1e10bef640 10 monclient: _finish_hunting 0 2024-03-13T07:33:52.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.727+0000 7f1e10bef640 1 monclient: found mon.noname-b 2024-03-13T07:33:52.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.727+0000 7f1e10bef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:52.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.727+0000 7f1e10bef640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T07:33:52.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.727+0000 7f1e10bef640 1 -- 172.21.15.195:0/267460572 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c0afa20 con 0x7f1e0c0b5320 2024-03-13T07:33:52.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.729+0000 7f1e10bef640 1 --2- 172.21.15.195:0/267460572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0c0b5320 0x7f1e0c0b0220 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1e000097d0 tx=0x7f1e0002b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3211ae28053b8638 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:52.734 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e0b7fe640 1 -- 172.21.15.195:0/267460572 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00031770 con 0x7f1e0c0b5320 2024-03-13T07:33:52.734 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e0b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:52.734 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e0b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:33:52.734 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e0b7fe640 10 monclient: dump: 2024-03-13T07:33:52.734 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:52.734 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:52.734 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.734 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.734 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:52.734 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:52.734 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:52.735 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:52.735 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:52.735 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:52.735 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e0b7fe640 10 monclient: _finish_auth 0 2024-03-13T07:33:52.735 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e0b7fe640 10 monclient: _check_auth_tickets 2024-03-13T07:33:52.735 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e0b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:52.735 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e0b7fe640 1 -- 172.21.15.195:0/267460572 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e00037070 con 0x7f1e0c0b5320 2024-03-13T07:33:52.735 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e0b7fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:52.735 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e0b7fe640 1 -- 172.21.15.195:0/267460572 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00031b30 con 0x7f1e0c0b5320 2024-03-13T07:33:52.735 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e12e53640 5 monclient: authenticate success, global_id 4142 2024-03-13T07:33:52.735 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e12e53640 10 monclient: get_monmap_and_config success 2024-03-13T07:33:52.735 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e12e53640 4 set_mon_vals no callback set 2024-03-13T07:33:52.736 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e0b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:52.736 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e0b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:33:52.736 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e0b7fe640 10 monclient: dump: 2024-03-13T07:33:52.736 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:52.736 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:52.736 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.736 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.736 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:52.736 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:52.736 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:52.736 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:52.736 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:52.736 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:52.737 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e12e53640 10 monclient: shutdown 2024-03-13T07:33:52.737 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e12e53640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:52.737 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e12e53640 1 -- 172.21.15.195:0/267460572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0c0b5320 msgr2=0x7f1e0c0b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:52.737 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e12e53640 1 --2- 172.21.15.195:0/267460572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0c0b5320 0x7f1e0c0b0220 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1e000097d0 tx=0x7f1e0002b810 comp rx=0 tx=0).stop 2024-03-13T07:33:52.737 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e12e53640 1 -- 172.21.15.195:0/267460572 shutdown_connections 2024-03-13T07:33:52.737 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e12e53640 1 --2- 172.21.15.195:0/267460572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0c0b5320 0x7f1e0c0b0220 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.737 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e12e53640 1 -- 172.21.15.195:0/267460572 >> 172.21.15.195:0/267460572 conn(0x7f1e0c06d2b0 msgr2=0x7f1e0c06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:52.737 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.733+0000 7f1e12e53640 1 -- 172.21.15.195:0/267460572 shutdown_connections 2024-03-13T07:33:52.737 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.734+0000 7f1e12e53640 1 -- 172.21.15.195:0/267460572 wait complete. 2024-03-13T07:33:52.742 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.734+0000 7f1e12e53640 10 monclient: build_initial_monmap 2024-03-13T07:33:52.743 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.734+0000 7f1e12e53640 10 monclient: monmap: 2024-03-13T07:33:52.743 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 0 2024-03-13T07:33:52.743 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:33:52.743 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 0.000000 2024-03-13T07:33:52.743 INFO:teuthology.orchestra.run.smithi195.stderr:created 0.000000 2024-03-13T07:33:52.743 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:52.743 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:52.743 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:33:52.743 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:33:52.743 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:33:52.743 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:52.743 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.734+0000 7f1e12e53640 1 librados: starting msgr at 2024-03-13T07:33:52.744 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.734+0000 7f1e12e53640 1 librados: starting objecter 2024-03-13T07:33:52.744 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.734+0000 7f1e12e53640 1 Processor -- start 2024-03-13T07:33:52.744 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 1 -- start start 2024-03-13T07:33:52.744 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 1 librados: setting wanted keys 2024-03-13T07:33:52.744 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 1 librados: calling monclient init 2024-03-13T07:33:52.744 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 10 monclient: init 2024-03-13T07:33:52.744 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:52.744 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 10 monclient: _add_conns ranks=[0,1,2] 2024-03-13T07:33:52.744 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e0c0b1ac0 0x7f1e0c1dafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:52.744 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 10 monclient(hunting): picked mon.noname-a con 0x7f1e0c0b1ac0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:33:52.745 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e0c0b5320 0x7f1e0c1409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:52.745 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 10 monclient(hunting): picked mon.noname-b con 0x7f1e0c0b5320 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:33:52.745 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0c13fd40 0x7f1e0c140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:52.745 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 10 monclient(hunting): picked mon.noname-c con 0x7f1e0c13fd40 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:52.745 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:52.745 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0b4980 con 0x7f1e0c0b1ac0 2024-03-13T07:33:52.745 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:52.745 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0b4800 con 0x7f1e0c0b5320 2024-03-13T07:33:52.746 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:52.746 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0b1f00 con 0x7f1e0c13fd40 2024-03-13T07:33:52.746 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:52.746 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.735+0000 7f1e12e53640 10 monclient(hunting): authenticate will time out at 1247.074164s 2024-03-13T07:33:52.746 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.736+0000 7f1e113f0640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0c13fd40 0x7f1e0c140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:52.746 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.736+0000 7f1e0bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e0c0b5320 0x7f1e0c1409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:52.746 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.736+0000 7f1e113f0640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0c13fd40 0x7f1e0c140f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.195:48830/0 (socket says 172.21.15.195:48830) 2024-03-13T07:33:52.746 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.736+0000 7f1e113f0640 1 -- 172.21.15.195:0/315456650 learned_addr learned my addr 172.21.15.195:0/315456650 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-13T07:33:52.746 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.736+0000 7f1e113f0640 10 monclient(hunting): get_auth_request con 0x7f1e0c13fd40 auth_method 0 2024-03-13T07:33:52.746 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.736+0000 7f1e113f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:52.746 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.736+0000 7f1e113f0640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:52.747 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.736+0000 7f1e113f0640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:52.747 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.736+0000 7f1e0bfff640 10 monclient(hunting): get_auth_request con 0x7f1e0c0b5320 auth_method 0 2024-03-13T07:33:52.747 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.736+0000 7f1e0bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:52.747 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.736+0000 7f1e0bfff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:52.747 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.736+0000 7f1e0bfff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:52.747 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.736+0000 7f1e113f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:52.747 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.736+0000 7f1e113f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:52.747 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.736+0000 7f1e113f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:52.747 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.736+0000 7f1e10bef640 1 --2- 172.21.15.195:0/315456650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e0c0b1ac0 0x7f1e0c1dafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:52.747 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.737+0000 7f1e0bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:52.747 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.737+0000 7f1e0bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:52.747 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.737+0000 7f1e0bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:52.747 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.737+0000 7f1e10bef640 10 monclient(hunting): get_auth_request con 0x7f1e0c0b1ac0 auth_method 0 2024-03-13T07:33:52.748 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.737+0000 7f1e10bef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:52.748 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.737+0000 7f1e10bef640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:52.748 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.737+0000 7f1e10bef640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:52.748 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.737+0000 7f1e10bef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:52.748 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.737+0000 7f1e10bef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:52.748 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.737+0000 7f1e10bef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:52.748 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.739+0000 7f1e113f0640 10 monclient(hunting): handle_auth_done global_id 4145 payload 824 2024-03-13T07:33:52.749 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.739+0000 7f1e113f0640 1 -- 172.21.15.195:0/315456650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e0c0b5320 msgr2=0x7f1e0c1409e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:52.749 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.739+0000 7f1e113f0640 1 --2- 172.21.15.195:0/315456650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e0c0b5320 0x7f1e0c1409e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.749 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.739+0000 7f1e113f0640 1 -- 172.21.15.195:0/315456650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e0c0b1ac0 msgr2=0x7f1e0c1dafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:52.749 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.739+0000 7f1e113f0640 1 --2- 172.21.15.195:0/315456650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e0c0b1ac0 0x7f1e0c1dafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:52.749 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.739+0000 7f1e113f0640 10 monclient: _finish_hunting 0 2024-03-13T07:33:52.749 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.739+0000 7f1e113f0640 1 monclient: found mon.noname-c 2024-03-13T07:33:52.750 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.739+0000 7f1e113f0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:52.750 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.739+0000 7f1e113f0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:33:52.750 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.739+0000 7f1e113f0640 1 -- 172.21.15.195:0/315456650 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c1db5d0 con 0x7f1e0c13fd40 2024-03-13T07:33:52.750 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.742+0000 7f1e113f0640 1 --2- 172.21.15.195:0/315456650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0c13fd40 0x7f1e0c140f20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400caa0 tx=0x7f1e0400cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:52.750 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.742+0000 7f1e097fa640 10 client.?.objecter ms_handle_connect 0x7f1e0c13fd40 2024-03-13T07:33:52.750 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.742+0000 7f1e097fa640 10 client.?.objecter resend_mon_ops 2024-03-13T07:33:52.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e097fa640 1 -- 172.21.15.195:0/315456650 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04017070 con 0x7f1e0c13fd40 2024-03-13T07:33:52.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e097fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:52.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e097fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:33:52.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e097fa640 10 monclient: dump: 2024-03-13T07:33:52.753 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:52.753 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:52.753 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.754 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.754 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:52.754 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:52.754 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:52.754 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:52.754 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:52.754 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:52.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e097fa640 10 monclient: _finish_auth 0 2024-03-13T07:33:52.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e097fa640 10 monclient: _check_auth_tickets 2024-03-13T07:33:52.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e097fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:52.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e097fa640 1 -- 172.21.15.195:0/315456650 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0400ac00 con 0x7f1e0c13fd40 2024-03-13T07:33:52.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e097fa640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:52.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e097fa640 1 -- 172.21.15.195:0/315456650 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04012400 con 0x7f1e0c13fd40 2024-03-13T07:33:52.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e12e53640 5 monclient: authenticate success, global_id 4145 2024-03-13T07:33:52.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e12e53640 10 monclient: _renew_subs 2024-03-13T07:33:52.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e12e53640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:52.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e12e53640 1 -- 172.21.15.195:0/315456650 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0c0ab140 con 0x7f1e0c13fd40 2024-03-13T07:33:52.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e0a7fc640 4 set_mon_vals no callback set 2024-03-13T07:33:52.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e097fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:52.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e097fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:33:52.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e097fa640 10 monclient: dump: 2024-03-13T07:33:52.755 INFO:teuthology.orchestra.run.smithi195.stderr:epoch 1 2024-03-13T07:33:52.756 INFO:teuthology.orchestra.run.smithi195.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:52.756 INFO:teuthology.orchestra.run.smithi195.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.756 INFO:teuthology.orchestra.run.smithi195.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:52.756 INFO:teuthology.orchestra.run.smithi195.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:52.756 INFO:teuthology.orchestra.run.smithi195.stderr:election_strategy: 1 2024-03-13T07:33:52.757 INFO:teuthology.orchestra.run.smithi195.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:52.757 INFO:teuthology.orchestra.run.smithi195.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:52.757 INFO:teuthology.orchestra.run.smithi195.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:52.757 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T07:33:52.757 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e12e53640 10 client.4145.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:33:52.757 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e12e53640 10 monclient: _renew_subs 2024-03-13T07:33:52.757 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e12e53640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:52.757 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e12e53640 1 -- 172.21.15.195:0/315456650 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c0ab700 con 0x7f1e0c13fd40 2024-03-13T07:33:52.757 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.746+0000 7f1e12e53640 1 librados: init done 2024-03-13T07:33:52.757 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.747+0000 7f1e12e53640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:52.758 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.747+0000 7f1e12e53640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:52.758 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.747+0000 7f1e12e53640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:52.758 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.747+0000 7f1e12e53640 1 -- 172.21.15.195:0/315456650 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0c04f000 con 0x7f1e0c13fd40 2024-03-13T07:33:52.758 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:52.756+0000 7fd30663e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-13T07:33:52.810 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:52.810+0000 7fb6b5844040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-13T07:33:52.843 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.843+0000 7f1e097fa640 1 -- 172.21.15.195:0/315456650 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1e0401a070 con 0x7f1e0c13fd40 2024-03-13T07:33:52.848 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.847+0000 7f1e097fa640 1 -- 172.21.15.195:0/315456650 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f1e040076d0 con 0x7f1e0c13fd40 2024-03-13T07:33:52.848 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.847+0000 7f1e097fa640 10 client.4145.objecter ms_dispatch 0x7f1e0c000b90 osd_map(9..9 src has 1..9) v4 2024-03-13T07:33:52.848 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.847+0000 7f1e097fa640 3 client.4145.objecter handle_osd_map got epochs [9,9] > 0 2024-03-13T07:33:52.848 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.847+0000 7f1e097fa640 3 client.4145.objecter handle_osd_map decoding full epoch 9 2024-03-13T07:33:52.848 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.847+0000 7f1e097fa640 20 client.4145.objecter dump_active .. 0 homeless 2024-03-13T07:33:52.848 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.847+0000 7f1e097fa640 1 -- 172.21.15.195:0/315456650 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1e04021020 con 0x7f1e0c13fd40 2024-03-13T07:33:52.848 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.847+0000 7f1e097fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:52.848 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.847+0000 7f1e097fa640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:33:52.872 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-13T07:33:52.872+0000 7fd30663e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-13T07:33:52.920 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.920+0000 7f1e12e53640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "19a39645-8664-463b-bae3-d7aa1026e5f9", "id": 7}] 2024-03-13T07:33:52.920 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.920+0000 7f1e12e53640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "19a39645-8664-463b-bae3-d7aa1026e5f9", "id": 7}] 2024-03-13T07:33:52.920 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.920+0000 7f1e12e53640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:52.920 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:52.920+0000 7f1e12e53640 1 -- 172.21.15.195:0/315456650 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "19a39645-8664-463b-bae3-d7aa1026e5f9", "id": 7} v 0) v1 -- 0x7f1e0c1dbb70 con 0x7f1e0c13fd40 2024-03-13T07:33:53.157 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.157+0000 7f1e097fa640 1 -- 172.21.15.195:0/315456650 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mgrmap(e 2) v1 ==== 45093+0+0 (secure 0 0 0) 0x7f1e04007070 con 0x7f1e0c13fd40 2024-03-13T07:33:53.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.360+0000 7f1e097fa640 1 -- 172.21.15.195:0/315456650 <== mon.2 v2:172.21.15.132:3301/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "19a39645-8664-463b-bae3-d7aa1026e5f9", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1e04020bb0 con 0x7f1e0c13fd40 2024-03-13T07:33:53.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.360+0000 7f1e097fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "19a39645-8664-463b-bae3-d7aa1026e5f9", "id": 7}] 2024-03-13T07:33:53.360 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.360+0000 7f1e097fa640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:33:53.360 INFO:teuthology.orchestra.run.smithi195.stdout:7 2024-03-13T07:33:53.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.360+0000 7f1e12e53640 10 librados: watch_flush enter 2024-03-13T07:33:53.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.361+0000 7f1e12e53640 10 librados: watch_flush exit 2024-03-13T07:33:53.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.361+0000 7f1e12e53640 20 client.4145.objecter shutdown clearing up homeless session... 2024-03-13T07:33:53.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.361+0000 7f1e12e53640 10 client.4145.objecter successfully canceled tick 2024-03-13T07:33:53.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.361+0000 7f1e12e53640 10 monclient: shutdown 2024-03-13T07:33:53.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.361+0000 7f1e12e53640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:53.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.361+0000 7f1e12e53640 1 -- 172.21.15.195:0/315456650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0c13fd40 msgr2=0x7f1e0c140f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:53.362 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.361+0000 7f1e12e53640 1 --2- 172.21.15.195:0/315456650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0c13fd40 0x7f1e0c140f20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400caa0 tx=0x7f1e0400cf60 comp rx=0 tx=0).stop 2024-03-13T07:33:53.363 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.361+0000 7f1e12e53640 1 -- 172.21.15.195:0/315456650 shutdown_connections 2024-03-13T07:33:53.363 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.361+0000 7f1e12e53640 1 --2- 172.21.15.195:0/315456650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0c13fd40 0x7f1e0c140f20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:53.363 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.361+0000 7f1e12e53640 1 --2- 172.21.15.195:0/315456650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e0c0b5320 0x7f1e0c1409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:53.363 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.361+0000 7f1e12e53640 1 --2- 172.21.15.195:0/315456650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e0c0b1ac0 0x7f1e0c1dafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:53.363 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.361+0000 7f1e12e53640 1 -- 172.21.15.195:0/315456650 >> 172.21.15.195:0/315456650 conn(0x7f1e0c06d2b0 msgr2=0x7f1e0c13e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:53.363 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.361+0000 7f1e12e53640 1 -- 172.21.15.195:0/315456650 shutdown_connections 2024-03-13T07:33:53.363 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.361+0000 7f1e12e53640 1 -- 172.21.15.195:0/315456650 wait complete. 2024-03-13T07:33:53.363 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-13T07:33:53.361+0000 7f1e12e53640 1 librados: shutdown 2024-03-13T07:33:53.371 DEBUG:tasks.ceph_manager:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-03-13T07:33:53.371 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-13T07:33:53.371 DEBUG:teuthology.orchestra.run.smithi132:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 2024-03-13T07:33:53.374 INFO:tasks.ceph.osd.0:Started 2024-03-13T07:33:53.374 DEBUG:tasks.ceph_manager:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-03-13T07:33:53.374 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-13T07:33:53.374 DEBUG:teuthology.orchestra.run.smithi132:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 2024-03-13T07:33:53.376 INFO:tasks.ceph.osd.1:Started 2024-03-13T07:33:53.377 DEBUG:tasks.ceph_manager:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-03-13T07:33:53.377 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-13T07:33:53.377 DEBUG:teuthology.orchestra.run.smithi132:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 2024-03-13T07:33:53.379 INFO:tasks.ceph.osd.2:Started 2024-03-13T07:33:53.379 DEBUG:tasks.ceph_manager:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-03-13T07:33:53.379 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-13T07:33:53.379 DEBUG:teuthology.orchestra.run.smithi132:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 2024-03-13T07:33:53.381 INFO:tasks.ceph.osd.3:Started 2024-03-13T07:33:53.382 DEBUG:tasks.ceph_manager:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-03-13T07:33:53.382 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-13T07:33:53.382 DEBUG:teuthology.orchestra.run.smithi195:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 2024-03-13T07:33:53.384 INFO:tasks.ceph.osd.4:Started 2024-03-13T07:33:53.384 DEBUG:tasks.ceph_manager:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-03-13T07:33:53.384 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-13T07:33:53.384 DEBUG:teuthology.orchestra.run.smithi195:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 2024-03-13T07:33:53.386 INFO:tasks.ceph.osd.5:Started 2024-03-13T07:33:53.386 DEBUG:tasks.ceph_manager:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-03-13T07:33:53.387 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-13T07:33:53.387 DEBUG:teuthology.orchestra.run.smithi195:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 2024-03-13T07:33:53.389 INFO:tasks.ceph.osd.6:Started 2024-03-13T07:33:53.389 DEBUG:tasks.ceph_manager:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-03-13T07:33:53.389 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-13T07:33:53.389 DEBUG:teuthology.orchestra.run.smithi195:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 2024-03-13T07:33:53.391 INFO:tasks.ceph.osd.7:Started 2024-03-13T07:33:53.392 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-13T07:33:53.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.552+0000 7faca4719640 10 monclient: get_monmap_and_config 2024-03-13T07:33:53.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.552+0000 7faca4719640 10 monclient: build_initial_monmap 2024-03-13T07:33:53.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.552+0000 7faca4719640 10 monclient: monmap: 2024-03-13T07:33:53.553 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:33:53.553 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:53.553 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:53.553260+0000 2024-03-13T07:33:53.553 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:53.553260+0000 2024-03-13T07:33:53.553 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:53.553 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:53.554 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:33:53.554 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:33:53.554 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:33:53.554 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:33:53.554 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:33:53.554 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:33:53.554 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.553+0000 7faca4719640 1 Processor -- start 2024-03-13T07:33:53.554 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.553+0000 7faca4719640 1 -- start start 2024-03-13T07:33:53.554 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.553+0000 7faca4719640 10 monclient: init 2024-03-13T07:33:53.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.554+0000 7faca4719640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:53.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.554+0000 7faca4719640 10 monclient: _add_conns ranks=[4,3,1,0,2] 2024-03-13T07:33:53.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.554+0000 7faca4719640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fac9c0b5320 0x7fac9c0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:53.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.554+0000 7faca4719640 10 monclient(hunting): picked mon.noname-c con 0x7fac9c0b5320 addr v2:172.21.15.195:3300/0 2024-03-13T07:33:53.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.554+0000 7faca4719640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fac9c0b1ac0 0x7fac9c0b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:53.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.554+0000 7faca4719640 10 monclient(hunting): picked mon.noname-a con 0x7fac9c0b1ac0 addr v2:172.21.15.132:3300/0 2024-03-13T07:33:53.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.554+0000 7faca4719640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fac9c13f630 addr v1:172.21.15.195:6789/0 2024-03-13T07:33:53.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.554+0000 7faca4719640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9c0b1150 con 0x7fac9c13f630 2024-03-13T07:33:53.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.554+0000 7faca4719640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:53.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.554+0000 7faca4719640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac9c0b13c0 con 0x7fac9c0b1ac0 2024-03-13T07:33:53.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.554+0000 7faca4719640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:53.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.554+0000 7faca4719640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac9c0b16a0 con 0x7fac9c0b5320 2024-03-13T07:33:53.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.554+0000 7faca4719640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:53.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.554+0000 7faca4719640 10 monclient(hunting): authenticate will time out at 1260.962237s 2024-03-13T07:33:53.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.554+0000 7faca24b5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fac9c0b5320 0x7fac9c0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:53.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.555+0000 7faca24b5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fac9c0b5320 0x7fac9c0b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:50894/0 (socket says 172.21.15.132:50894) 2024-03-13T07:33:53.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.555+0000 7faca24b5640 1 -- 172.21.15.132:0/2461051930 learned_addr learned my addr 172.21.15.132:0/2461051930 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:33:53.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.555+0000 7faca24b5640 10 monclient(hunting): get_auth_request con 0x7fac9c0b5320 auth_method 0 2024-03-13T07:33:53.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.555+0000 7faca24b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:53.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.555+0000 7faca24b5640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:53.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.555+0000 7faca24b5640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:53.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.556+0000 7faca24b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:53.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.556+0000 7faca24b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:53.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.556+0000 7faca24b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:53.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.558+0000 7faca24b5640 10 monclient(hunting): handle_auth_done global_id 4153 payload 274 2024-03-13T07:33:53.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.559+0000 7faca24b5640 1 -- 172.21.15.132:0/2461051930 >> v2:172.21.15.132:3300/0 conn(0x7fac9c0b1ac0 msgr2=0x7fac9c0b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:33:53.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.559+0000 7faca24b5640 1 --2- 172.21.15.132:0/2461051930 >> v2:172.21.15.132:3300/0 conn(0x7fac9c0b1ac0 0x7fac9c0b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:53.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.559+0000 7faca24b5640 1 -- 172.21.15.132:0/2461051930 >> v1:172.21.15.195:6789/0 conn(0x7fac9c13f630 legacy=0x7fac9c13fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:53.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.559+0000 7faca24b5640 10 monclient: _finish_hunting 0 2024-03-13T07:33:53.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.559+0000 7faca24b5640 1 monclient: found mon.noname-c 2024-03-13T07:33:53.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.559+0000 7faca24b5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:53.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.559+0000 7faca24b5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T07:33:53.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.559+0000 7faca24b5640 1 -- 172.21.15.132:0/2461051930 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c0afa20 con 0x7fac9c0b5320 2024-03-13T07:33:53.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.563+0000 7faca24b5640 1 --2- 172.21.15.132:0/2461051930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac9c0b5320 0x7fac9c0b0220 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0b90f0 tx=0x7fac8c02b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c48608bb654f1ba server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:53.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.570+0000 7faca14b3640 1 -- 172.21.15.132:0/2461051930 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c0317f0 con 0x7fac9c0b5320 2024-03-13T07:33:53.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.570+0000 7faca14b3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:53.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.570+0000 7faca14b3640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:33:53.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.570+0000 7faca14b3640 10 monclient: dump: 2024-03-13T07:33:53.570 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:33:53.570 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:53.570 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:53.571 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:53.571 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:53.571 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:53.571 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:53.571 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:53.571 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:53.571 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:33:53.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.570+0000 7faca14b3640 10 monclient: _finish_auth 0 2024-03-13T07:33:53.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.570+0000 7faca14b3640 10 monclient: _check_auth_tickets 2024-03-13T07:33:53.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.570+0000 7faca14b3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:53.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.570+0000 7faca4719640 5 monclient: authenticate success, global_id 4153 2024-03-13T07:33:53.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.570+0000 7faca4719640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:33:53.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.573+0000 7faca14b3640 1 -- 172.21.15.132:0/2461051930 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c037070 con 0x7fac9c0b5320 2024-03-13T07:33:53.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.573+0000 7faca14b3640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:53.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.573+0000 7faca14b3640 1 -- 172.21.15.132:0/2461051930 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c031bb0 con 0x7fac9c0b5320 2024-03-13T07:33:53.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.573+0000 7faca14b3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:53.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.573+0000 7faca14b3640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:33:53.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.573+0000 7faca14b3640 10 monclient: dump: 2024-03-13T07:33:53.574 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:33:53.574 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:53.574 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:53.574 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:53.574 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:53.574 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:53.574 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:53.574 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:53.575 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:53.575 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:33:53.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.573+0000 7faca4719640 10 monclient: get_monmap_and_config success 2024-03-13T07:33:53.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.573+0000 7faca4719640 4 set_mon_vals no callback set 2024-03-13T07:33:53.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.574+0000 7faca4719640 10 monclient: shutdown 2024-03-13T07:33:53.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.574+0000 7faca4719640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:53.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.574+0000 7faca4719640 1 -- 172.21.15.132:0/2461051930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac9c0b5320 msgr2=0x7fac9c0b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:53.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.574+0000 7faca4719640 1 --2- 172.21.15.132:0/2461051930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac9c0b5320 0x7fac9c0b0220 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0b90f0 tx=0x7fac8c02b890 comp rx=0 tx=0).stop 2024-03-13T07:33:53.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.574+0000 7faca4719640 1 -- 172.21.15.132:0/2461051930 shutdown_connections 2024-03-13T07:33:53.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.574+0000 7faca4719640 1 --2- 172.21.15.132:0/2461051930 >> v2:172.21.15.132:3300/0 conn(0x7fac9c0b1ac0 0x7fac9c0b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:53.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.574+0000 7faca4719640 1 --2- 172.21.15.132:0/2461051930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac9c0b5320 0x7fac9c0b0220 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:53.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.574+0000 7faca4719640 1 -- 172.21.15.132:0/2461051930 >> 172.21.15.132:0/2461051930 conn(0x7fac9c06d2b0 msgr2=0x7fac9c06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:53.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.574+0000 7faca4719640 1 -- 172.21.15.132:0/2461051930 shutdown_connections 2024-03-13T07:33:53.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.575+0000 7faca4719640 1 -- 172.21.15.132:0/2461051930 wait complete. 2024-03-13T07:33:53.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.575+0000 7faca4719640 10 monclient: build_initial_monmap 2024-03-13T07:33:53.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.575+0000 7faca4719640 10 monclient: monmap: 2024-03-13T07:33:53.579 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:33:53.579 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:33:53.579 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:33:53.579 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:33:53.579 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:53.579 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:53.579 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:33:53.579 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:33:53.580 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:33:53.580 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:33:53.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.576+0000 7faca4719640 1 librados: starting msgr at 2024-03-13T07:33:53.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.576+0000 7faca4719640 1 librados: starting objecter 2024-03-13T07:33:53.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.576+0000 7faca4719640 1 Processor -- start 2024-03-13T07:33:53.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.576+0000 7faca4719640 1 -- start start 2024-03-13T07:33:53.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.576+0000 7faca4719640 1 librados: setting wanted keys 2024-03-13T07:33:53.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.576+0000 7faca4719640 1 librados: calling monclient init 2024-03-13T07:33:53.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.576+0000 7faca4719640 10 monclient: init 2024-03-13T07:33:53.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca4719640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:53.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca4719640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T07:33:53.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca4719640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac9c0b1ac0 0x7fac9c141c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:53.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca4719640 10 monclient(hunting): picked mon.noname-a con 0x7fac9c0b1ac0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:33:53.581 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca4719640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac9c0b5320 0x7fac9c1df6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:53.581 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca4719640 10 monclient(hunting): picked mon.noname-c con 0x7fac9c0b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:53.581 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca4719640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac9c13f630 0x7fac9c1dfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:53.581 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca4719640 10 monclient(hunting): picked mon.noname-b con 0x7fac9c13f630 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:33:53.581 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca4719640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:53.581 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca4719640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c0b4980 con 0x7fac9c0b1ac0 2024-03-13T07:33:53.581 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca4719640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:53.581 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca4719640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c0b4800 con 0x7fac9c13f630 2024-03-13T07:33:53.581 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca4719640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:53.581 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca4719640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c14cef0 con 0x7fac9c0b5320 2024-03-13T07:33:53.581 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca4719640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:53.581 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca4719640 10 monclient(hunting): authenticate will time out at 1260.985190s 2024-03-13T07:33:53.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.577+0000 7faca1cb4640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac9c0b5320 0x7fac9c1df6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:53.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.578+0000 7faca1cb4640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac9c0b5320 0x7fac9c1df6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:57928/0 (socket says 172.21.15.132:57928) 2024-03-13T07:33:53.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.578+0000 7faca1cb4640 1 -- 172.21.15.132:0/4188248406 learned_addr learned my addr 172.21.15.132:0/4188248406 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:33:53.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.578+0000 7faca1cb4640 10 monclient(hunting): get_auth_request con 0x7fac9c0b5320 auth_method 0 2024-03-13T07:33:53.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.578+0000 7faca1cb4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:53.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.578+0000 7faca1cb4640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:53.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.578+0000 7faca1cb4640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:53.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.579+0000 7faca1cb4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:53.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.579+0000 7faca1cb4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:53.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.579+0000 7faca1cb4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:53.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.581+0000 7faca2cb6640 1 --2- 172.21.15.132:0/4188248406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac9c13f630 0x7fac9c1dfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:53.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.581+0000 7faca2cb6640 10 monclient(hunting): get_auth_request con 0x7fac9c13f630 auth_method 0 2024-03-13T07:33:53.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.581+0000 7faca2cb6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:53.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.581+0000 7faca2cb6640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:53.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.581+0000 7faca2cb6640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:53.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.581+0000 7faca2cb6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:53.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.581+0000 7faca2cb6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:53.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.581+0000 7faca2cb6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:53.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.583+0000 7faca1cb4640 10 monclient(hunting): handle_auth_done global_id 4166 payload 824 2024-03-13T07:33:53.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.583+0000 7faca1cb4640 1 -- 172.21.15.132:0/4188248406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac9c13f630 msgr2=0x7fac9c1dfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:53.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.583+0000 7faca1cb4640 1 --2- 172.21.15.132:0/4188248406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac9c13f630 0x7fac9c1dfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:53.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.583+0000 7faca1cb4640 1 -- 172.21.15.132:0/4188248406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac9c0b1ac0 msgr2=0x7fac9c141c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:53.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.583+0000 7faca1cb4640 1 --2- 172.21.15.132:0/4188248406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac9c0b1ac0 0x7fac9c141c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:53.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.583+0000 7faca1cb4640 10 monclient: _finish_hunting 0 2024-03-13T07:33:53.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.583+0000 7faca1cb4640 1 monclient: found mon.noname-c 2024-03-13T07:33:53.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.583+0000 7faca1cb4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:53.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.583+0000 7faca1cb4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:33:53.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.583+0000 7faca1cb4640 1 -- 172.21.15.132:0/4188248406 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c143360 con 0x7fac9c0b5320 2024-03-13T07:33:53.586 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.587+0000 7faca1cb4640 1 --2- 172.21.15.132:0/4188248406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac9c0b5320 0x7fac9c1df6f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fac9800b4d0 tx=0x7fac9800b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:53.586 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.587+0000 7fac92ffd640 10 client.?.objecter ms_handle_connect 0x7fac9c0b5320 2024-03-13T07:33:53.586 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.587+0000 7fac92ffd640 10 client.?.objecter resend_mon_ops 2024-03-13T07:33:53.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7fac92ffd640 1 -- 172.21.15.132:0/4188248406 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98014070 con 0x7fac9c0b5320 2024-03-13T07:33:53.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7fac92ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:53.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7fac92ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:33:53.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7fac92ffd640 10 monclient: dump: 2024-03-13T07:33:53.595 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:33:53.595 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:53.595 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:53.595 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:53.595 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:53.595 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:53.595 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:53.595 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:53.595 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:53.595 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:33:53.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7fac92ffd640 10 monclient: _finish_auth 0 2024-03-13T07:33:53.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7fac92ffd640 10 monclient: _check_auth_tickets 2024-03-13T07:33:53.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7fac92ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:53.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7faca4719640 5 monclient: authenticate success, global_id 4166 2024-03-13T07:33:53.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7faca4719640 10 monclient: _renew_subs 2024-03-13T07:33:53.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7faca4719640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:53.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7faca4719640 1 -- 172.21.15.132:0/4188248406 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9c137370 con 0x7fac9c0b5320 2024-03-13T07:33:53.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7faca4719640 10 client.4166.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:33:53.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7faca4719640 10 monclient: _renew_subs 2024-03-13T07:33:53.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7faca4719640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:53.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7faca4719640 1 -- 172.21.15.132:0/4188248406 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9c142330 con 0x7fac9c0b5320 2024-03-13T07:33:53.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7faca4719640 1 librados: init done 2024-03-13T07:33:53.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7fac92ffd640 1 -- 172.21.15.132:0/4188248406 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac98010870 con 0x7fac9c0b5320 2024-03-13T07:33:53.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7fac92ffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:53.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.592+0000 7faca0cb2640 4 set_mon_vals no callback set 2024-03-13T07:33:53.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.593+0000 7fac92ffd640 1 -- 172.21.15.132:0/4188248406 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98010c60 con 0x7fac9c0b5320 2024-03-13T07:33:53.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.593+0000 7fac92ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:53.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.594+0000 7fac92ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:33:53.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.594+0000 7fac92ffd640 10 monclient: dump: 2024-03-13T07:33:53.597 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:33:53.597 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:53.597 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:53.597 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:53.597 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:53.597 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:53.597 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:53.598 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:53.598 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:53.598 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:33:53.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.594+0000 7faca4719640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:53.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.594+0000 7faca4719640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:53.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.594+0000 7faca4719640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:53.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.594+0000 7faca4719640 1 -- 172.21.15.132:0/4188248406 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac84005080 con 0x7fac9c0b5320 2024-03-13T07:33:53.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.839+0000 7fac92ffd640 1 -- 172.21.15.132:0/4188248406 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 2) v1 ==== 45093+0+0 (secure 0 0 0) 0x7fac98017070 con 0x7fac9c0b5320 2024-03-13T07:33:53.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.845+0000 7fac92ffd640 1 -- 172.21.15.132:0/4188248406 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fac9800fdb0 con 0x7fac9c0b5320 2024-03-13T07:33:53.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.845+0000 7fac92ffd640 10 client.4166.objecter ms_dispatch 0x7fac9c000b90 osd_map(10..10 src has 1..10) v4 2024-03-13T07:33:53.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.845+0000 7fac92ffd640 3 client.4166.objecter handle_osd_map got epochs [10,10] > 0 2024-03-13T07:33:53.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.845+0000 7fac92ffd640 3 client.4166.objecter handle_osd_map decoding full epoch 10 2024-03-13T07:33:53.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.846+0000 7fac92ffd640 20 client.4166.objecter dump_active .. 0 homeless 2024-03-13T07:33:53.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.846+0000 7fac92ffd640 1 -- 172.21.15.132:0/4188248406 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fac98022c50 con 0x7fac9c0b5320 2024-03-13T07:33:53.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.846+0000 7fac92ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:53.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.846+0000 7fac92ffd640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:33:53.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.969+0000 7faca4719640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:33:53.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.969+0000 7faca4719640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:33:53.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.969+0000 7faca4719640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:53.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:53.969+0000 7faca4719640 1 -- 172.21.15.132:0/4188248406 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac84005640 con 0x7fac9c0b5320 2024-03-13T07:33:54.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.054+0000 7fac92ffd640 1 -- 172.21.15.132:0/4188248406 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fac980239a0 con 0x7fac9c0b5320 2024-03-13T07:33:54.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.054+0000 7fac92ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:33:54.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.054+0000 7fac92ffd640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:33:54.055 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:33:54.055 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":10,"fsid":"74e4bc2f-6a5e-4423-a8c8-807970a2f96c","created":"2024-03-13T07:33:47.494293+0000","modified":"2024-03-13T07:33:53.171597+0000","last_up_change":"0.000000","last_in_change":"2024-03-13T07:33:53.171597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0edb216c-bc1e-4865-aafe-218611f3eb09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6be0ae6b-951e-4279-ba0a-9b05b6e6e85e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"18008b7f-cfe7-4450-a3cf-aab63ebbe99d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"411b6066-4b9f-40e7-b92d-4ba914eab63e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1a719246-ae04-42b0-bd9c-fc318dc57ecb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d9ee991-241b-4507-ba2f-e3b5e2844d1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"af8b34fd-927b-4860-971b-9fddf699a0dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"19a39645-8664-463b-bae3-d7aa1026e5f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T07:33:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.056+0000 7faca4719640 10 librados: watch_flush enter 2024-03-13T07:33:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.056+0000 7faca4719640 10 librados: watch_flush exit 2024-03-13T07:33:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.056+0000 7faca4719640 20 client.4166.objecter shutdown clearing up homeless session... 2024-03-13T07:33:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.056+0000 7faca4719640 10 client.4166.objecter successfully canceled tick 2024-03-13T07:33:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.056+0000 7faca4719640 10 monclient: shutdown 2024-03-13T07:33:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.056+0000 7faca4719640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.056+0000 7faca4719640 1 -- 172.21.15.132:0/4188248406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac9c0b5320 msgr2=0x7fac9c1df6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.056+0000 7faca4719640 1 --2- 172.21.15.132:0/4188248406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac9c0b5320 0x7fac9c1df6f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fac9800b4d0 tx=0x7fac9800b990 comp rx=0 tx=0).stop 2024-03-13T07:33:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.056+0000 7faca4719640 1 -- 172.21.15.132:0/4188248406 shutdown_connections 2024-03-13T07:33:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.056+0000 7faca4719640 1 --2- 172.21.15.132:0/4188248406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac9c13f630 0x7fac9c1dfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.057+0000 7faca4719640 1 --2- 172.21.15.132:0/4188248406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac9c0b5320 0x7fac9c1df6f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.057+0000 7faca4719640 1 --2- 172.21.15.132:0/4188248406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac9c0b1ac0 0x7fac9c141c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.057+0000 7faca4719640 1 -- 172.21.15.132:0/4188248406 >> 172.21.15.132:0/4188248406 conn(0x7fac9c06d2b0 msgr2=0x7fac9c0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.057+0000 7faca4719640 1 -- 172.21.15.132:0/4188248406 shutdown_connections 2024-03-13T07:33:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.057+0000 7faca4719640 1 -- 172.21.15.132:0/4188248406 wait complete. 2024-03-13T07:33:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:33:54.057+0000 7faca4719640 1 librados: shutdown 2024-03-13T07:33:54.073 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-13T07:33:54.074 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-13T07:33:54.507 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:54.508+0000 7fb656551640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-13T07:33:54.590 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T07:33:54.591+0000 7fb656551640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-13T07:33:54.880 DEBUG:teuthology.orchestra.run.smithi132:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-13T07:33:55.075 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.075+0000 7f529a1cd640 10 monclient: get_monmap_and_config 2024-03-13T07:33:55.075 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.075+0000 7f529a1cd640 10 monclient: build_initial_monmap 2024-03-13T07:33:55.075 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.075+0000 7f529a1cd640 10 monclient: monmap: 2024-03-13T07:33:55.075 INFO:teuthology.misc.health.smithi132.stderr:epoch 0 2024-03-13T07:33:55.075 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:55.075 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:55.076147+0000 2024-03-13T07:33:55.076 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:55.076147+0000 2024-03-13T07:33:55.076 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:55.076 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:55.076 INFO:teuthology.misc.health.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:33:55.076 INFO:teuthology.misc.health.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:33:55.076 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:33:55.076 INFO:teuthology.misc.health.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:33:55.076 INFO:teuthology.misc.health.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:33:55.076 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:33:55.079 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.076+0000 7f529a1cd640 1 Processor -- start 2024-03-13T07:33:55.079 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.076+0000 7f529a1cd640 1 -- start start 2024-03-13T07:33:55.079 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.076+0000 7f529a1cd640 10 monclient: init 2024-03-13T07:33:55.079 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.077+0000 7f529a1cd640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:55.079 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.077+0000 7f529a1cd640 10 monclient: _add_conns ranks=[2,1,3,0,4] 2024-03-13T07:33:55.079 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.077+0000 7f529a1cd640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52940b5110 0x7f529413f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:55.079 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.077+0000 7f529a1cd640 10 monclient(hunting): picked mon.noname-b con 0x7f52940b5110 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:55.079 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.077+0000 7f529a1cd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f52940b22c0 addr v1:172.21.15.195:6789/0 2024-03-13T07:33:55.079 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.077+0000 7f529a1cd640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f529413fdb0 0x7f5294142d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:55.079 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.077+0000 7f529a1cd640 10 monclient(hunting): picked mon.noname-a con 0x7f529413fdb0 addr v2:172.21.15.132:3300/0 2024-03-13T07:33:55.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.077+0000 7f529a1cd640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5294143250 con 0x7f52940b22c0 2024-03-13T07:33:55.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.077+0000 7f529a1cd640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:55.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.077+0000 7f529a1cd640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52940b54f0 con 0x7f529413fdb0 2024-03-13T07:33:55.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.077+0000 7f529a1cd640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:55.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.077+0000 7f529a1cd640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5294140240 con 0x7f52940b5110 2024-03-13T07:33:55.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.077+0000 7f529a1cd640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:55.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.077+0000 7f529a1cd640 10 monclient(hunting): authenticate will time out at 1262.485773s 2024-03-13T07:33:55.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.078+0000 7f52937fe640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52940b5110 0x7f529413f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:55.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.078+0000 7f5292ffd640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f52940b22c0 0x7f5294142600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.132:42618/0 (socket says 172.21.15.132:42618) 2024-03-13T07:33:55.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.078+0000 7f5292ffd640 1 -- 172.21.15.132:0/397001774 learned_addr learned my addr 172.21.15.132:0/397001774 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:33:55.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.078+0000 7f52937fe640 10 monclient(hunting): get_auth_request con 0x7f52940b5110 auth_method 0 2024-03-13T07:33:55.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.078+0000 7f52937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:55.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.078+0000 7f52937fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:55.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.078+0000 7f52937fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:55.081 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.079+0000 7f52937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:55.081 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.079+0000 7f52937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:55.081 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.079+0000 7f52937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:55.082 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.081+0000 7f52937fe640 10 monclient(hunting): handle_auth_done global_id 4169 payload 274 2024-03-13T07:33:55.082 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.081+0000 7f52937fe640 1 -- 172.21.15.132:0/397001774 >> v2:172.21.15.132:3300/0 conn(0x7f529413fdb0 msgr2=0x7f5294142d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:33:55.082 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.081+0000 7f52937fe640 1 --2- 172.21.15.132:0/397001774 >> v2:172.21.15.132:3300/0 conn(0x7f529413fdb0 0x7f5294142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:55.082 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.081+0000 7f52937fe640 1 -- 172.21.15.132:0/397001774 >> v1:172.21.15.195:6789/0 conn(0x7f52940b22c0 legacy=0x7f5294142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:55.082 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.081+0000 7f52937fe640 10 monclient: _finish_hunting 0 2024-03-13T07:33:55.082 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.081+0000 7f52937fe640 1 monclient: found mon.noname-b 2024-03-13T07:33:55.082 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.081+0000 7f52937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:55.082 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.081+0000 7f52937fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T07:33:55.082 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.081+0000 7f52937fe640 1 -- 172.21.15.132:0/397001774 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52940b4750 con 0x7f52940b5110 2024-03-13T07:33:55.083 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.084+0000 7f52937fe640 1 --2- 172.21.15.132:0/397001774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52940b5110 0x7f529413f870 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f52840097a0 tx=0x7f528402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bc6d2b3c54d6f5d server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:55.091 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.089+0000 7f52927fc640 1 -- 172.21.15.132:0/397001774 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284031780 con 0x7f52940b5110 2024-03-13T07:33:55.091 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.089+0000 7f52927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:55.091 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.089+0000 7f52927fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:33:55.092 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.089+0000 7f52927fc640 10 monclient: dump: 2024-03-13T07:33:55.092 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:33:55.092 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:55.092 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:55.092 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:55.092 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:55.092 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:55.092 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:55.092 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:55.092 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:55.092 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:33:55.092 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.089+0000 7f52927fc640 10 monclient: _finish_auth 0 2024-03-13T07:33:55.092 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.089+0000 7f52927fc640 10 monclient: _check_auth_tickets 2024-03-13T07:33:55.092 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.089+0000 7f52927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:55.092 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.089+0000 7f52927fc640 1 -- 172.21.15.132:0/397001774 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5284037070 con 0x7f52940b5110 2024-03-13T07:33:55.093 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.089+0000 7f52927fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:55.093 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.089+0000 7f52927fc640 1 -- 172.21.15.132:0/397001774 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284031b40 con 0x7f52940b5110 2024-03-13T07:33:55.093 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.089+0000 7f52927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:55.093 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.089+0000 7f52927fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:33:55.093 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.089+0000 7f52927fc640 10 monclient: dump: 2024-03-13T07:33:55.093 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:33:55.093 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:55.093 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:55.093 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:55.093 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:55.093 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:55.093 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:55.093 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:55.093 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:55.094 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:33:55.094 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.089+0000 7f529a1cd640 5 monclient: authenticate success, global_id 4169 2024-03-13T07:33:55.094 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.089+0000 7f529a1cd640 10 monclient: get_monmap_and_config success 2024-03-13T07:33:55.094 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.089+0000 7f529a1cd640 4 set_mon_vals no callback set 2024-03-13T07:33:55.094 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.090+0000 7f529a1cd640 10 monclient: shutdown 2024-03-13T07:33:55.094 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.090+0000 7f529a1cd640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:55.094 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.090+0000 7f529a1cd640 1 -- 172.21.15.132:0/397001774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52940b5110 msgr2=0x7f529413f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:55.094 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.090+0000 7f529a1cd640 1 --2- 172.21.15.132:0/397001774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52940b5110 0x7f529413f870 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f52840097a0 tx=0x7f528402b820 comp rx=0 tx=0).stop 2024-03-13T07:33:55.094 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.091+0000 7f529a1cd640 1 -- 172.21.15.132:0/397001774 shutdown_connections 2024-03-13T07:33:55.094 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.091+0000 7f529a1cd640 1 --2- 172.21.15.132:0/397001774 >> v2:172.21.15.132:3300/0 conn(0x7f529413fdb0 0x7f5294142d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:55.094 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.091+0000 7f529a1cd640 1 --2- 172.21.15.132:0/397001774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52940b5110 0x7f529413f870 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:55.094 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.091+0000 7f529a1cd640 1 -- 172.21.15.132:0/397001774 >> 172.21.15.132:0/397001774 conn(0x7f5294072780 msgr2=0x7f5294072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:55.094 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.091+0000 7f529a1cd640 1 -- 172.21.15.132:0/397001774 shutdown_connections 2024-03-13T07:33:55.094 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.092+0000 7f529a1cd640 1 -- 172.21.15.132:0/397001774 wait complete. 2024-03-13T07:33:55.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.093+0000 7f529a1cd640 10 monclient: build_initial_monmap 2024-03-13T07:33:55.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.093+0000 7f529a1cd640 10 monclient: monmap: 2024-03-13T07:33:55.099 INFO:teuthology.misc.health.smithi132.stderr:epoch 0 2024-03-13T07:33:55.100 INFO:teuthology.misc.health.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:33:55.100 INFO:teuthology.misc.health.smithi132.stderr:last_changed 0.000000 2024-03-13T07:33:55.100 INFO:teuthology.misc.health.smithi132.stderr:created 0.000000 2024-03-13T07:33:55.100 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:33:55.100 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:55.100 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:33:55.100 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:33:55.100 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:33:55.100 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:33:55.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.093+0000 7f529a1cd640 1 librados: starting msgr at 2024-03-13T07:33:55.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.093+0000 7f529a1cd640 1 librados: starting objecter 2024-03-13T07:33:55.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.093+0000 7f529a1cd640 1 Processor -- start 2024-03-13T07:33:55.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.093+0000 7f529a1cd640 1 -- start start 2024-03-13T07:33:55.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.093+0000 7f529a1cd640 1 librados: setting wanted keys 2024-03-13T07:33:55.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.093+0000 7f529a1cd640 1 librados: calling monclient init 2024-03-13T07:33:55.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.093+0000 7f529a1cd640 10 monclient: init 2024-03-13T07:33:55.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.094+0000 7f529a1cd640 10 monclient: _reopen_session rank -1 2024-03-13T07:33:55.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.094+0000 7f529a1cd640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T07:33:55.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.094+0000 7f529a1cd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52940b22c0 0x7f52941dfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:55.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.094+0000 7f529a1cd640 10 monclient(hunting): picked mon.noname-a con 0x7f52940b22c0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:33:55.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.094+0000 7f529a1cd640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52940b5110 0x7f52941e0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:55.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.094+0000 7f529a1cd640 10 monclient(hunting): picked mon.noname-c con 0x7f52940b5110 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:33:55.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.094+0000 7f529a1cd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f529413fdb0 0x7f52941e0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:55.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.094+0000 7f529a1cd640 10 monclient(hunting): picked mon.noname-b con 0x7f529413fdb0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:33:55.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.094+0000 7f529a1cd640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:55.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.094+0000 7f529a1cd640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52940a9eb0 con 0x7f52940b22c0 2024-03-13T07:33:55.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.094+0000 7f529a1cd640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:55.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.094+0000 7f529a1cd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52940aa1a0 con 0x7f529413fdb0 2024-03-13T07:33:55.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.094+0000 7f529a1cd640 10 monclient(hunting): start opening mon connection 2024-03-13T07:33:55.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.094+0000 7f529a1cd640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529414ce50 con 0x7f52940b5110 2024-03-13T07:33:55.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.094+0000 7f529a1cd640 10 monclient(hunting): _renew_subs 2024-03-13T07:33:55.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.094+0000 7f529a1cd640 10 monclient(hunting): authenticate will time out at 1262.503025s 2024-03-13T07:33:55.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.095+0000 7f5292ffd640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52940b5110 0x7f52941e0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:55.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.095+0000 7f5293fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f529413fdb0 0x7f52941e0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:55.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.095+0000 7f5292ffd640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52940b5110 0x7f52941e0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:57946/0 (socket says 172.21.15.132:57946) 2024-03-13T07:33:55.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.095+0000 7f5292ffd640 1 -- 172.21.15.132:0/978686182 learned_addr learned my addr 172.21.15.132:0/978686182 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:33:55.103 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.095+0000 7f5292ffd640 10 monclient(hunting): get_auth_request con 0x7f52940b5110 auth_method 0 2024-03-13T07:33:55.103 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.095+0000 7f5292ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:55.103 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.095+0000 7f5292ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:55.103 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.095+0000 7f5292ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:55.103 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.096+0000 7f5293fff640 10 monclient(hunting): get_auth_request con 0x7f529413fdb0 auth_method 0 2024-03-13T07:33:55.103 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.096+0000 7f5293fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:33:55.103 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.096+0000 7f5293fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:33:55.103 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.096+0000 7f5293fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:33:55.103 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.096+0000 7f5292ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:55.103 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.096+0000 7f5292ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:55.103 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.096+0000 7f5292ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:55.104 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.097+0000 7f5293fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:33:55.104 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.097+0000 7f5293fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:33:55.104 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.097+0000 7f5293fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:33:55.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.100+0000 7f5292ffd640 10 monclient(hunting): handle_auth_done global_id 4172 payload 824 2024-03-13T07:33:55.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.100+0000 7f5292ffd640 1 -- 172.21.15.132:0/978686182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f529413fdb0 msgr2=0x7f52941e0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:55.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.100+0000 7f5292ffd640 1 --2- 172.21.15.132:0/978686182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f529413fdb0 0x7f52941e0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:55.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.100+0000 7f5292ffd640 1 -- 172.21.15.132:0/978686182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52940b22c0 msgr2=0x7f52941dfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:55.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.100+0000 7f5292ffd640 1 --2- 172.21.15.132:0/978686182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52940b22c0 0x7f52941dfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:55.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.100+0000 7f5292ffd640 10 monclient: _finish_hunting 0 2024-03-13T07:33:55.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.100+0000 7f5292ffd640 1 monclient: found mon.noname-c 2024-03-13T07:33:55.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.100+0000 7f5292ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:33:55.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.100+0000 7f5292ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:33:55.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.100+0000 7f5292ffd640 1 -- 172.21.15.132:0/978686182 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529406faa0 con 0x7f52940b5110 2024-03-13T07:33:55.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.104+0000 7f5292ffd640 1 --2- 172.21.15.132:0/978686182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52940b5110 0x7f52941e0420 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f528800c400 tx=0x7f528800c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:55.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.104+0000 7f527bfff640 10 client.?.objecter ms_handle_connect 0x7f52940b5110 2024-03-13T07:33:55.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.104+0000 7f527bfff640 10 client.?.objecter resend_mon_ops 2024-03-13T07:33:55.111 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.109+0000 7f527bfff640 1 -- 172.21.15.132:0/978686182 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288017070 con 0x7f52940b5110 2024-03-13T07:33:55.111 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.109+0000 7f527bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:55.111 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.109+0000 7f527bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:33:55.111 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.109+0000 7f527bfff640 10 monclient: dump: 2024-03-13T07:33:55.111 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:33:55.111 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:55.112 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:55.112 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:55.112 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:55.112 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:55.112 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:55.112 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:55.112 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:55.112 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:33:55.112 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.109+0000 7f527bfff640 10 monclient: _finish_auth 0 2024-03-13T07:33:55.112 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.109+0000 7f527bfff640 10 monclient: _check_auth_tickets 2024-03-13T07:33:55.112 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.109+0000 7f527bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:33:55.112 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.109+0000 7f529a1cd640 5 monclient: authenticate success, global_id 4172 2024-03-13T07:33:55.112 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.109+0000 7f529a1cd640 10 monclient: _renew_subs 2024-03-13T07:33:55.112 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.109+0000 7f529a1cd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:55.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.109+0000 7f529a1cd640 1 -- 172.21.15.132:0/978686182 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52941475f0 con 0x7f52940b5110 2024-03-13T07:33:55.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.109+0000 7f527bfff640 1 -- 172.21.15.132:0/978686182 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52880074e0 con 0x7f52940b5110 2024-03-13T07:33:55.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.110+0000 7f529a1cd640 10 client.4172.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:33:55.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.110+0000 7f529a1cd640 10 monclient: _renew_subs 2024-03-13T07:33:55.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.110+0000 7f529a1cd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:55.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.110+0000 7f529a1cd640 1 -- 172.21.15.132:0/978686182 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529406f580 con 0x7f52940b5110 2024-03-13T07:33:55.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.110+0000 7f529a1cd640 1 librados: init done 2024-03-13T07:33:55.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.110+0000 7f527bfff640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:33:55.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.110+0000 7f5291ffb640 4 set_mon_vals no callback set 2024-03-13T07:33:55.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.111+0000 7f527bfff640 1 -- 172.21.15.132:0/978686182 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52880059a0 con 0x7f52940b5110 2024-03-13T07:33:55.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.111+0000 7f527bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:33:55.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.111+0000 7f527bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:33:55.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.111+0000 7f527bfff640 10 monclient: dump: 2024-03-13T07:33:55.113 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:33:55.113 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:33:55.114 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:55.114 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:33:55.114 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:33:55.114 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:33:55.114 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:33:55.114 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:33:55.114 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:33:55.114 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:33:55.114 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.111+0000 7f5279ffb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:55.114 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.111+0000 7f5279ffb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:55.114 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.111+0000 7f5279ffb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:55.114 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.111+0000 7f5279ffb640 1 -- 172.21.15.132:0/978686182 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529406f780 con 0x7f52940b5110 2024-03-13T07:33:55.371 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.371+0000 7f527bfff640 1 -- 172.21.15.132:0/978686182 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 3) v1 ==== 89974+0+0 (secure 0 0 0) 0x7f528801a070 con 0x7f52940b5110 2024-03-13T07:33:55.371 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.371+0000 7f527bfff640 1 --2- 172.21.15.132:0/978686182 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f528006d3b0 0x7f528006f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:33:55.371 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.372+0000 7f52937fe640 1 --2- 172.21.15.132:0/978686182 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f528006d3b0 0x7f528006f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:33:55.371 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.372+0000 7f52937fe640 10 monclient: get_auth_request con 0x7f528006d3b0 auth_method 0 2024-03-13T07:33:55.372 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.372+0000 7f527bfff640 10 client.4172.objecter ms_handle_connect 0x7f528006d3b0 2024-03-13T07:33:55.372 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.372+0000 7f52937fe640 1 --2- 172.21.15.132:0/978686182 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f528006d3b0 0x7f528006f870 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f52840097a0 tx=0x7f5284030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:33:55.596 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.595+0000 7f527bfff640 1 -- 172.21.15.132:0/978686182 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f5288057620 con 0x7f52940b5110 2024-03-13T07:33:55.596 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.595+0000 7f527bfff640 10 client.4172.objecter ms_dispatch 0x7f5294000b90 osd_map(10..10 src has 1..10) v4 2024-03-13T07:33:55.596 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.595+0000 7f527bfff640 3 client.4172.objecter handle_osd_map got epochs [10,10] > 0 2024-03-13T07:33:55.596 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.595+0000 7f527bfff640 3 client.4172.objecter handle_osd_map decoding full epoch 10 2024-03-13T07:33:55.596 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.595+0000 7f527bfff640 20 client.4172.objecter dump_active .. 0 homeless 2024-03-13T07:33:55.596 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.595+0000 7f527bfff640 1 -- 172.21.15.132:0/978686182 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f529406f780 con 0x7f52940b5110 2024-03-13T07:33:55.596 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.595+0000 7f527bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:33:55.596 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.596+0000 7f527bfff640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:33:55.956 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.956+0000 7f5279ffb640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:33:55.956 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.956+0000 7f5279ffb640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:33:55.956 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.956+0000 7f5279ffb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:33:55.956 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:55.956+0000 7f5279ffb640 1 -- 172.21.15.132:0/978686182 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5294068990 con 0x7f52940b5110 2024-03-13T07:33:56.126 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.125+0000 7f527bfff640 1 -- 172.21.15.132:0/978686182 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mgrmap(e 4) v1 ==== 90080+0+0 (secure 0 0 0) 0x7f5288055000 con 0x7f52940b5110 2024-03-13T07:33:56.160 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.160+0000 7f527bfff640 1 -- 172.21.15.132:0/978686182 <== mon.2 v2:172.21.15.132:3301/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f528803e470 con 0x7f52940b5110 2024-03-13T07:33:56.161 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.160+0000 7f527bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:33:56.161 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.160+0000 7f527bfff640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:33:56.161 INFO:teuthology.misc.health.smithi132.stdout: 2024-03-13T07:33:56.161 INFO:teuthology.misc.health.smithi132.stdout:{"epoch":10,"fsid":"74e4bc2f-6a5e-4423-a8c8-807970a2f96c","created":"2024-03-13T07:33:47.494293+0000","modified":"2024-03-13T07:33:53.171597+0000","last_up_change":"0.000000","last_in_change":"2024-03-13T07:33:53.171597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0edb216c-bc1e-4865-aafe-218611f3eb09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6be0ae6b-951e-4279-ba0a-9b05b6e6e85e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"18008b7f-cfe7-4450-a3cf-aab63ebbe99d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"411b6066-4b9f-40e7-b92d-4ba914eab63e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1a719246-ae04-42b0-bd9c-fc318dc57ecb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d9ee991-241b-4507-ba2f-e3b5e2844d1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"af8b34fd-927b-4860-971b-9fddf699a0dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"19a39645-8664-463b-bae3-d7aa1026e5f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T07:33:56.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.163+0000 7f5279ffb640 10 librados: watch_flush enter 2024-03-13T07:33:56.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.163+0000 7f5279ffb640 10 librados: watch_flush exit 2024-03-13T07:33:56.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.163+0000 7f5279ffb640 20 client.4172.objecter shutdown clearing up homeless session... 2024-03-13T07:33:56.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.163+0000 7f5279ffb640 10 client.4172.objecter successfully canceled tick 2024-03-13T07:33:56.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.164+0000 7f5279ffb640 1 -- 172.21.15.132:0/978686182 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f528006d3b0 msgr2=0x7f528006f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:56.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.164+0000 7f5279ffb640 1 --2- 172.21.15.132:0/978686182 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f528006d3b0 0x7f528006f870 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f52840097a0 tx=0x7f5284030000 comp rx=0 tx=0).stop 2024-03-13T07:33:56.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.164+0000 7f5279ffb640 10 monclient: shutdown 2024-03-13T07:33:56.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.164+0000 7f5279ffb640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:33:56.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.164+0000 7f5279ffb640 1 -- 172.21.15.132:0/978686182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52940b5110 msgr2=0x7f52941e0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:33:56.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.164+0000 7f5279ffb640 1 --2- 172.21.15.132:0/978686182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52940b5110 0x7f52941e0420 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f528800c400 tx=0x7f528800c8c0 comp rx=0 tx=0).stop 2024-03-13T07:33:56.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.164+0000 7f5279ffb640 1 -- 172.21.15.132:0/978686182 shutdown_connections 2024-03-13T07:33:56.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.164+0000 7f5279ffb640 1 --2- 172.21.15.132:0/978686182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f529413fdb0 0x7f52941e0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:56.165 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.164+0000 7f5279ffb640 1 --2- 172.21.15.132:0/978686182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52940b5110 0x7f52941e0420 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:56.165 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.164+0000 7f5279ffb640 1 --2- 172.21.15.132:0/978686182 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f528006d3b0 0x7f528006f870 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:56.165 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.164+0000 7f5279ffb640 1 --2- 172.21.15.132:0/978686182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52940b22c0 0x7f52941dfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:33:56.165 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.164+0000 7f5279ffb640 1 -- 172.21.15.132:0/978686182 >> 172.21.15.132:0/978686182 conn(0x7f5294072780 msgr2=0x7f52940b76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:33:56.165 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.164+0000 7f5279ffb640 1 -- 172.21.15.132:0/978686182 shutdown_connections 2024-03-13T07:33:56.165 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.164+0000 7f5279ffb640 1 -- 172.21.15.132:0/978686182 wait complete. 2024-03-13T07:33:56.165 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:33:56.164+0000 7f5279ffb640 1 librados: shutdown 2024-03-13T07:33:56.181 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-13T07:34:00.835 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:07.365 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:00.837 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:07.386 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:00.897 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:07.446 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:00.902 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:07.432 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:00.958 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:07.506 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:00.985 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:07.511 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:01.018 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:07.566 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:01.050 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:07.576 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:01.278 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:07.816 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:01.341 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:07.879 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:01.387 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:07.911 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:01.428 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:07.971 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:01.442 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-03-13T07:34:01.437+0000 5dc3480 -1 Falling back to public interface 2024-03-13T07:34:01.444 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T07:34:01.440+0000 5dc3480 -1 Falling back to public interface 2024-03-13T07:34:01.453 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:07.977 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:01.494 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:08.037 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:01.498 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:08.027 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:01.562 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:08.091 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:01.592 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-03-13T07:34:01.586+0000 5dc3480 -1 Falling back to public interface 2024-03-13T07:34:01.611 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:34:01.605+0000 5dc3480 -1 Falling back to public interface 2024-03-13T07:34:01.889 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:01.884+0000 5dc3480 -1 Falling back to public interface 2024-03-13T07:34:01.932 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:08.481 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:01.961 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:08.491 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.002 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-03-13T07:34:01.997+0000 5dc3480 -1 Falling back to public interface 2024-03-13T07:34:02.050 INFO:tasks.ceph.osd.0.smithi132.stderr:2024-03-13T07:34:02.044+0000 5dc3480 -1 Falling back to public interface 2024-03-13T07:34:02.100 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:08.648 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.137 INFO:tasks.ceph.osd.2.smithi132.stderr:2024-03-13T07:34:02.131+0000 5dc3480 -1 Falling back to public interface 2024-03-13T07:34:02.156 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:08.682 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.321 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:08.870 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.357 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:08.888 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.404 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:08.943 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.491 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:09.039 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.516 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:09.040 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.547 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:09.073 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.556 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:09.099 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.647 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:09.177 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.652 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:09.201 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.683 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:09.211 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.781 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:09.329 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.799 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:09.338 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.844 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:09.370 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.913 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:09.442 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.916 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:09.440 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.948 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:09.497 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.959 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:09.502 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:02.987 DEBUG:teuthology.orchestra.run.smithi132:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-13T07:34:03.049 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:09.597 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.076 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.076+0000 7f4c82024640 10 monclient: get_monmap_and_config 2024-03-13T07:34:03.076 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.076+0000 7f4c82024640 10 monclient: build_initial_monmap 2024-03-13T07:34:03.076 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.076+0000 7f4c82024640 10 monclient: monmap: 2024-03-13T07:34:03.076 INFO:teuthology.misc.health.smithi132.stderr:epoch 0 2024-03-13T07:34:03.076 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:03.077 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:34:03.077330+0000 2024-03-13T07:34:03.077 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:34:03.077330+0000 2024-03-13T07:34:03.077 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:03.077 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:03.077 INFO:teuthology.misc.health.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:03.077 INFO:teuthology.misc.health.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:03.077 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:03.077 INFO:teuthology.misc.health.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:03.077 INFO:teuthology.misc.health.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:03.077 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:03.079 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.077+0000 7f4c82024640 1 Processor -- start 2024-03-13T07:34:03.079 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.077+0000 7f4c82024640 1 -- start start 2024-03-13T07:34:03.079 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.077+0000 7f4c82024640 10 monclient: init 2024-03-13T07:34:03.079 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.077+0000 7f4c82024640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:03.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.077+0000 7f4c82024640 10 monclient: _add_conns ranks=[4,1,3,0,2] 2024-03-13T07:34:03.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.077+0000 7f4c82024640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4c7c0b5110 0x7f4c7c13f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:03.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.077+0000 7f4c82024640 10 monclient(hunting): picked mon.noname-c con 0x7f4c7c0b5110 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:03.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c82024640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4c7c0b22c0 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:03.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c82024640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4c7c13fdb0 0x7f4c7c142d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:03.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c82024640 10 monclient(hunting): picked mon.noname-a con 0x7f4c7c13fdb0 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:03.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c82024640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c7c140220 con 0x7f4c7c0b22c0 2024-03-13T07:34:03.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c82024640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:03.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c82024640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7c0b54f0 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c82024640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:03.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c82024640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7c143250 con 0x7f4c7c0b5110 2024-03-13T07:34:03.080 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c82024640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:03.081 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c82024640 10 monclient(hunting): authenticate will time out at 1270.486155s 2024-03-13T07:34:03.081 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c7b7fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4c7c0b5110 0x7f4c7c13f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:03.081 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c7bfff640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4c7c13fdb0 0x7f4c7c142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:03.081 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c7affd640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f4c7c0b22c0 0x7f4c7c142600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.132:46506/0 (socket says 172.21.15.132:46506) 2024-03-13T07:34:03.081 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c7affd640 1 -- 172.21.15.132:0/77606717 learned_addr learned my addr 172.21.15.132:0/77606717 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:34:03.081 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c7bfff640 10 monclient(hunting): get_auth_request con 0x7f4c7c13fdb0 auth_method 0 2024-03-13T07:34:03.081 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:03.081 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c7bfff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:03.081 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.078+0000 7f4c7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:03.081 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:09.607 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.083 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.079+0000 7f4c7b7fe640 10 monclient(hunting): get_auth_request con 0x7f4c7c0b5110 auth_method 0 2024-03-13T07:34:03.083 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.079+0000 7f4c7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:03.083 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.079+0000 7f4c7b7fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:03.083 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.079+0000 7f4c7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:03.083 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.079+0000 7f4c7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:03.083 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.079+0000 7f4c7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:03.083 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.079+0000 7f4c7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:03.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.080+0000 7f4c7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:03.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.080+0000 7f4c7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:03.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.080+0000 7f4c7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:03.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.081+0000 7f4c7bfff640 10 monclient(hunting): handle_auth_done global_id 4185 payload 274 2024-03-13T07:34:03.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.081+0000 7f4c7bfff640 1 -- 172.21.15.132:0/77606717 >> v2:172.21.15.195:3300/0 conn(0x7f4c7c0b5110 msgr2=0x7f4c7c13f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:03.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.081+0000 7f4c7bfff640 1 --2- 172.21.15.132:0/77606717 >> v2:172.21.15.195:3300/0 conn(0x7f4c7c0b5110 0x7f4c7c13f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:03.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.081+0000 7f4c7bfff640 1 -- 172.21.15.132:0/77606717 >> v1:172.21.15.195:6789/0 conn(0x7f4c7c0b22c0 legacy=0x7f4c7c142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:03.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.081+0000 7f4c7bfff640 10 monclient: _finish_hunting 0 2024-03-13T07:34:03.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.081+0000 7f4c7bfff640 1 monclient: found mon.noname-a 2024-03-13T07:34:03.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.081+0000 7f4c7bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:03.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.081+0000 7f4c7bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:03.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.081+0000 7f4c7bfff640 1 -- 172.21.15.132:0/77606717 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7c0b4750 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.082+0000 7f4c7bfff640 1 --2- 172.21.15.132:0/77606717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c7c13fdb0 0x7f4c7c142d10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0b8ee0 tx=0x7f4c7402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24df282a5a0fc342 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:03.086 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.086+0000 7f4c7a7fc640 1 -- 172.21.15.132:0/77606717 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c74039070 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.086 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.086+0000 7f4c7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:03.090 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.086+0000 7f4c7a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:03.090 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.086+0000 7f4c7a7fc640 10 monclient: dump: 2024-03-13T07:34:03.090 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:03.090 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:03.090 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:03.090 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:03.090 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:03.091 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:03.091 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:03.091 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:03.091 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:03.091 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:03.091 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.086+0000 7f4c7a7fc640 10 monclient: _finish_auth 0 2024-03-13T07:34:03.091 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.086+0000 7f4c7a7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:34:03.091 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.086+0000 7f4c7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:03.091 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.086+0000 7f4c7a7fc640 1 -- 172.21.15.132:0/77606717 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7402cbf0 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.091 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.086+0000 7f4c7a7fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:03.091 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.086+0000 7f4c7a7fc640 1 -- 172.21.15.132:0/77606717 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c74034440 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.091 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.086+0000 7f4c82024640 5 monclient: authenticate success, global_id 4185 2024-03-13T07:34:03.091 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.086+0000 7f4c82024640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:03.091 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.086+0000 7f4c82024640 4 set_mon_vals no callback set 2024-03-13T07:34:03.092 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.086+0000 7f4c7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:03.092 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.086+0000 7f4c7a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:03.092 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.086+0000 7f4c7a7fc640 10 monclient: dump: 2024-03-13T07:34:03.092 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:03.092 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:03.092 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:03.092 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:03.092 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:03.092 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:03.092 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:03.092 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:03.092 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:03.092 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:03.096 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.087+0000 7f4c82024640 10 monclient: shutdown 2024-03-13T07:34:03.096 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.087+0000 7f4c82024640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:03.096 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.087+0000 7f4c82024640 1 -- 172.21.15.132:0/77606717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c7c13fdb0 msgr2=0x7f4c7c142d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:03.097 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.087+0000 7f4c82024640 1 --2- 172.21.15.132:0/77606717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c7c13fdb0 0x7f4c7c142d10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0b8ee0 tx=0x7f4c7402be40 comp rx=0 tx=0).stop 2024-03-13T07:34:03.097 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.087+0000 7f4c82024640 1 -- 172.21.15.132:0/77606717 shutdown_connections 2024-03-13T07:34:03.097 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.087+0000 7f4c82024640 1 --2- 172.21.15.132:0/77606717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c7c13fdb0 0x7f4c7c142d10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:03.097 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.087+0000 7f4c82024640 1 --2- 172.21.15.132:0/77606717 >> v2:172.21.15.195:3300/0 conn(0x7f4c7c0b5110 0x7f4c7c13f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:03.097 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.087+0000 7f4c82024640 1 -- 172.21.15.132:0/77606717 >> 172.21.15.132:0/77606717 conn(0x7f4c7c072780 msgr2=0x7f4c7c072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:03.097 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.087+0000 7f4c82024640 1 -- 172.21.15.132:0/77606717 shutdown_connections 2024-03-13T07:34:03.097 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.087+0000 7f4c82024640 1 -- 172.21.15.132:0/77606717 wait complete. 2024-03-13T07:34:03.097 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.088+0000 7f4c82024640 10 monclient: build_initial_monmap 2024-03-13T07:34:03.097 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.088+0000 7f4c82024640 10 monclient: monmap: 2024-03-13T07:34:03.097 INFO:teuthology.misc.health.smithi132.stderr:epoch 0 2024-03-13T07:34:03.097 INFO:teuthology.misc.health.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:03.097 INFO:teuthology.misc.health.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:03.097 INFO:teuthology.misc.health.smithi132.stderr:created 0.000000 2024-03-13T07:34:03.098 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:03.098 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:03.098 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:03.098 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:03.098 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:03.098 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:03.098 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.088+0000 7f4c82024640 1 librados: starting msgr at 2024-03-13T07:34:03.098 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.088+0000 7f4c82024640 1 librados: starting objecter 2024-03-13T07:34:03.098 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.088+0000 7f4c82024640 1 Processor -- start 2024-03-13T07:34:03.098 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.088+0000 7f4c82024640 1 -- start start 2024-03-13T07:34:03.098 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.088+0000 7f4c82024640 1 librados: setting wanted keys 2024-03-13T07:34:03.098 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.088+0000 7f4c82024640 1 librados: calling monclient init 2024-03-13T07:34:03.098 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.088+0000 7f4c82024640 10 monclient: init 2024-03-13T07:34:03.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.089+0000 7f4c82024640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:03.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.089+0000 7f4c82024640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T07:34:03.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.089+0000 7f4c82024640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c7c0b22c0 0x7f4c7c149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:03.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.089+0000 7f4c82024640 10 monclient(hunting): picked mon.noname-a con 0x7f4c7c0b22c0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:03.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.089+0000 7f4c82024640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c7c0b5110 0x7f4c7c14a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:03.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.089+0000 7f4c82024640 10 monclient(hunting): picked mon.noname-c con 0x7f4c7c0b5110 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:03.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.089+0000 7f4c82024640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c7c13fdb0 0x7f4c7c146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:03.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.089+0000 7f4c82024640 10 monclient(hunting): picked mon.noname-b con 0x7f4c7c13fdb0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:03.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.089+0000 7f4c82024640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:03.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.089+0000 7f4c82024640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0a9eb0 con 0x7f4c7c0b22c0 2024-03-13T07:34:03.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.089+0000 7f4c82024640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:03.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.089+0000 7f4c82024640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0aa1a0 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.089+0000 7f4c82024640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:03.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.089+0000 7f4c82024640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c14ce50 con 0x7f4c7c0b5110 2024-03-13T07:34:03.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.089+0000 7f4c82024640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:03.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.089+0000 7f4c82024640 10 monclient(hunting): authenticate will time out at 1270.497653s 2024-03-13T07:34:03.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c7c13fdb0 0x7f4c7c146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:03.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7affd640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c7c0b5110 0x7f4c7c14a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:03.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c7c0b22c0 0x7f4c7c149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:03.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c7c13fdb0 0x7f4c7c146e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:59656/0 (socket says 172.21.15.132:59656) 2024-03-13T07:34:03.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7bfff640 1 -- 172.21.15.132:0/1867563084 learned_addr learned my addr 172.21.15.132:0/1867563084 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:03.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7bfff640 10 monclient(hunting): get_auth_request con 0x7f4c7c13fdb0 auth_method 0 2024-03-13T07:34:03.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:03.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7bfff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:03.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:03.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7affd640 10 monclient(hunting): get_auth_request con 0x7f4c7c0b5110 auth_method 0 2024-03-13T07:34:03.100 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:03.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7affd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:03.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7affd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:03.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7b7fe640 10 monclient(hunting): get_auth_request con 0x7f4c7c0b22c0 auth_method 0 2024-03-13T07:34:03.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:03.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7b7fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:03.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:03.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:03.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:03.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.090+0000 7f4c7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:03.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.091+0000 7f4c7affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:03.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.091+0000 7f4c7affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:03.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.091+0000 7f4c7affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:03.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.091+0000 7f4c7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:03.101 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.091+0000 7f4c7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:03.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.091+0000 7f4c7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:03.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.093+0000 7f4c7bfff640 10 monclient(hunting): handle_auth_done global_id 4192 payload 824 2024-03-13T07:34:03.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.093+0000 7f4c7bfff640 1 -- 172.21.15.132:0/1867563084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c7c0b5110 msgr2=0x7f4c7c14a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:03.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.093+0000 7f4c7bfff640 1 --2- 172.21.15.132:0/1867563084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c7c0b5110 0x7f4c7c14a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:03.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.093+0000 7f4c7bfff640 1 -- 172.21.15.132:0/1867563084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c7c0b22c0 msgr2=0x7f4c7c149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:03.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.093+0000 7f4c7bfff640 1 --2- 172.21.15.132:0/1867563084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c7c0b22c0 0x7f4c7c149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:03.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.093+0000 7f4c7bfff640 10 monclient: _finish_hunting 0 2024-03-13T07:34:03.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.093+0000 7f4c7bfff640 1 monclient: found mon.noname-b 2024-03-13T07:34:03.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.093+0000 7f4c7bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:03.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.093+0000 7f4c7bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:03.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.093+0000 7f4c7bfff640 1 -- 172.21.15.132:0/1867563084 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7c14b630 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.102 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.095+0000 7f4c7bfff640 1 --2- 172.21.15.132:0/1867563084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c7c13fdb0 0x7f4c7c146e20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009e80 tx=0x7f4c74004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:03.103 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.095+0000 7f4c63fff640 10 client.?.objecter ms_handle_connect 0x7f4c7c13fdb0 2024-03-13T07:34:03.103 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.095+0000 7f4c63fff640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:03.104 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c63fff640 1 -- 172.21.15.132:0/1867563084 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c74039050 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c63fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:03.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c63fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:03.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c63fff640 10 monclient: dump: 2024-03-13T07:34:03.105 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:03.105 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:03.105 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:03.105 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:03.105 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:03.105 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:03.105 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:03.105 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:03.105 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:03.106 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:03.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c63fff640 10 monclient: _finish_auth 0 2024-03-13T07:34:03.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c63fff640 10 monclient: _check_auth_tickets 2024-03-13T07:34:03.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c63fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:03.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c63fff640 1 -- 172.21.15.132:0/1867563084 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c74045070 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c82024640 5 monclient: authenticate success, global_id 4192 2024-03-13T07:34:03.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c82024640 10 monclient: _renew_subs 2024-03-13T07:34:03.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c82024640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:03.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c82024640 1 -- 172.21.15.132:0/1867563084 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c7c14b910 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c63fff640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:03.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c63fff640 1 -- 172.21.15.132:0/1867563084 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c74009450 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c63fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:03.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c63fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:03.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c63fff640 10 monclient: dump: 2024-03-13T07:34:03.107 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:03.107 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:03.107 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:03.107 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:03.107 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:03.107 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:03.107 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:03.107 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:03.107 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:03.107 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:03.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c79ffb640 4 set_mon_vals no callback set 2024-03-13T07:34:03.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c82024640 10 client.4192.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:03.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c82024640 10 monclient: _renew_subs 2024-03-13T07:34:03.108 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c82024640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:03.108 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c82024640 1 -- 172.21.15.132:0/1867563084 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7c14bd30 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.108 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.098+0000 7f4c82024640 1 librados: init done 2024-03-13T07:34:03.108 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.100+0000 7f4c82024640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:03.108 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.100+0000 7f4c82024640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:03.108 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.100+0000 7f4c82024640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:03.108 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.100+0000 7f4c82024640 1 -- 172.21.15.132:0/1867563084 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7c068990 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.116 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:09.642 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.117 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.117+0000 7f4c63fff640 1 -- 172.21.15.132:0/1867563084 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 4) v1 ==== 90080+0+0 (secure 0 0 0) 0x7f4c7403e850 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.117 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.117+0000 7f4c63fff640 1 --2- 172.21.15.132:0/1867563084 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4c6806d4d0 0x7f4c6806f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:03.117 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.118+0000 7f4c7b7fe640 1 --2- 172.21.15.132:0/1867563084 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4c6806d4d0 0x7f4c6806f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:03.118 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.118+0000 7f4c7b7fe640 10 monclient: get_auth_request con 0x7f4c6806d4d0 auth_method 0 2024-03-13T07:34:03.118 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.118+0000 7f4c7b7fe640 1 --2- 172.21.15.132:0/1867563084 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4c6806d4d0 0x7f4c6806f990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009b40 tx=0x7f4c70009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:03.118 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.118+0000 7f4c63fff640 10 client.4192.objecter ms_handle_connect 0x7f4c6806d4d0 2024-03-13T07:34:03.119 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.120+0000 7f4c63fff640 1 -- 172.21.15.132:0/1867563084 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4c7407b3a0 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.119 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.120+0000 7f4c63fff640 10 client.4192.objecter ms_dispatch 0x7f4c7c000b90 osd_map(10..10 src has 1..10) v4 2024-03-13T07:34:03.119 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.120+0000 7f4c63fff640 3 client.4192.objecter handle_osd_map got epochs [10,10] > 0 2024-03-13T07:34:03.120 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.120+0000 7f4c63fff640 3 client.4192.objecter handle_osd_map decoding full epoch 10 2024-03-13T07:34:03.120 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.120+0000 7f4c63fff640 20 client.4192.objecter dump_active .. 0 homeless 2024-03-13T07:34:03.134 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:09.672 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.178 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:09.708 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.225 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:09.749 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.242 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:09.791 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.297 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:09.837 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.316 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:09.863 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.357 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.357+0000 7f4c63fff640 1 -- 172.21.15.132:0/1867563084 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f4c7407c710 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.357 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.357+0000 7f4c63fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:03.357 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.357+0000 7f4c63fff640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:03.380 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:09.906 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.428 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:09.966 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.434 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:09.963 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.442 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:09.973 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.500 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:10.024 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.531 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:10.080 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.583 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:10.131 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.584 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:10.126 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.591 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.592+0000 7f4c82024640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:03.592 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.592+0000 7f4c82024640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:03.592 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.592+0000 7f4c82024640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:03.592 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.592+0000 7f4c82024640 1 -- 172.21.15.132:0/1867563084 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c7c06db90 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.635 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.636+0000 7f4c63fff640 1 -- 172.21.15.132:0/1867563084 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f4c740807b0 con 0x7f4c7c13fdb0 2024-03-13T07:34:03.636 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.636+0000 7f4c63fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:03.636 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.636+0000 7f4c63fff640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:03.636 INFO:teuthology.misc.health.smithi132.stdout: 2024-03-13T07:34:03.636 INFO:teuthology.misc.health.smithi132.stdout:{"epoch":10,"fsid":"74e4bc2f-6a5e-4423-a8c8-807970a2f96c","created":"2024-03-13T07:33:47.494293+0000","modified":"2024-03-13T07:33:53.171597+0000","last_up_change":"0.000000","last_in_change":"2024-03-13T07:33:53.171597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0edb216c-bc1e-4865-aafe-218611f3eb09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6be0ae6b-951e-4279-ba0a-9b05b6e6e85e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"18008b7f-cfe7-4450-a3cf-aab63ebbe99d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"411b6066-4b9f-40e7-b92d-4ba914eab63e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1a719246-ae04-42b0-bd9c-fc318dc57ecb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d9ee991-241b-4507-ba2f-e3b5e2844d1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"af8b34fd-927b-4860-971b-9fddf699a0dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"19a39645-8664-463b-bae3-d7aa1026e5f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T07:34:03.639 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 10 librados: watch_flush enter 2024-03-13T07:34:03.639 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 10 librados: watch_flush exit 2024-03-13T07:34:03.639 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 20 client.4192.objecter shutdown clearing up homeless session... 2024-03-13T07:34:03.639 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 10 client.4192.objecter successfully canceled tick 2024-03-13T07:34:03.639 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 1 -- 172.21.15.132:0/1867563084 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4c6806d4d0 msgr2=0x7f4c6806f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:03.639 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 1 --2- 172.21.15.132:0/1867563084 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4c6806d4d0 0x7f4c6806f990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009b40 tx=0x7f4c70009190 comp rx=0 tx=0).stop 2024-03-13T07:34:03.639 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 10 monclient: shutdown 2024-03-13T07:34:03.639 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:03.639 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 1 -- 172.21.15.132:0/1867563084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c7c13fdb0 msgr2=0x7f4c7c146e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:03.639 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 1 --2- 172.21.15.132:0/1867563084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c7c13fdb0 0x7f4c7c146e20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009e80 tx=0x7f4c74004a40 comp rx=0 tx=0).stop 2024-03-13T07:34:03.639 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 1 -- 172.21.15.132:0/1867563084 shutdown_connections 2024-03-13T07:34:03.640 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 1 --2- 172.21.15.132:0/1867563084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c7c13fdb0 0x7f4c7c146e20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:03.640 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 1 --2- 172.21.15.132:0/1867563084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c7c0b5110 0x7f4c7c14a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:03.640 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 1 --2- 172.21.15.132:0/1867563084 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4c6806d4d0 0x7f4c6806f990 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:03.640 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 1 --2- 172.21.15.132:0/1867563084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c7c0b22c0 0x7f4c7c149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:03.640 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 1 -- 172.21.15.132:0/1867563084 >> 172.21.15.132:0/1867563084 conn(0x7f4c7c072780 msgr2=0x7f4c7c0b5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:03.640 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 1 -- 172.21.15.132:0/1867563084 shutdown_connections 2024-03-13T07:34:03.640 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 1 -- 172.21.15.132:0/1867563084 wait complete. 2024-03-13T07:34:03.640 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:03.638+0000 7f4c82024640 1 librados: shutdown 2024-03-13T07:34:03.648 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-13T07:34:03.662 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:10.188 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.711 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:10.241 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.726 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:10.265 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.759 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:10.287 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.781 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:10.305 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.829 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:10.378 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.850 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:10.398 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.876 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:10.419 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.927 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:10.452 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:03.975 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:10.506 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.016 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:10.554 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.054 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:10.578 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.055 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:10.583 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.117 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:10.665 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.123 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:10.672 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.165 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:10.708 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.190 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:10.716 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.245 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:10.775 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.304 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:10.843 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.327 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:10.851 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.351 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:10.880 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.385 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:10.933 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.413 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:10.962 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.452 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:10.994 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.456 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:10.982 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.599 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:11.123 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.603 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:11.142 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.642 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:11.171 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.745 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:11.288 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.805 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:11.335 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.856 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:11.387 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.866 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:11.390 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.898 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:11.437 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.945 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:11.473 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.963 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:11.511 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.974 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:11.524 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:04.995 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:11.544 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:05.001 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:11.527 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:05.008 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:11.557 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:05.034 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:11.577 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:05.043 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:11.569 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:05.236 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:11.764 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:05.360 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:11.883 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:05.391 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:11.914 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:05.460 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:11.999 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:05.491 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:12.030 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:05.583 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:12.126 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:05.640 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:12.183 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:05.824 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:12.352 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:05.855 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:12.383 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:10.151 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:16.681 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:10.406 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:16.955 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:10.455 DEBUG:teuthology.orchestra.run.smithi132:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-13T07:34:10.471 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:17.019 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:10.546 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.545+0000 7f1515193640 10 monclient: get_monmap_and_config 2024-03-13T07:34:10.547 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.545+0000 7f1515193640 10 monclient: build_initial_monmap 2024-03-13T07:34:10.547 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.545+0000 7f1515193640 10 monclient: monmap: 2024-03-13T07:34:10.547 INFO:teuthology.misc.health.smithi132.stderr:epoch 0 2024-03-13T07:34:10.547 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:10.547 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:34:10.547621+0000 2024-03-13T07:34:10.547 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:34:10.547621+0000 2024-03-13T07:34:10.547 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:10.547 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:10.547 INFO:teuthology.misc.health.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:10.547 INFO:teuthology.misc.health.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:10.547 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:10.547 INFO:teuthology.misc.health.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:10.548 INFO:teuthology.misc.health.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:10.548 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:10.548 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.546+0000 7f1515193640 1 Processor -- start 2024-03-13T07:34:10.548 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.546+0000 7f1515193640 1 -- start start 2024-03-13T07:34:10.548 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.546+0000 7f1515193640 10 monclient: init 2024-03-13T07:34:10.550 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.547+0000 7f1515193640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:10.550 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.547+0000 7f1515193640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2024-03-13T07:34:10.550 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.547+0000 7f1515193640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15100b5110 0x7f151013f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:10.550 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.547+0000 7f1515193640 10 monclient(hunting): picked mon.noname-b con 0x7f15100b5110 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:10.551 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.547+0000 7f1515193640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f15100b22c0 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:10.551 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.547+0000 7f1515193640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f151013fdb0 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:10.551 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.547+0000 7f1515193640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15100b4750 con 0x7f15100b22c0 2024-03-13T07:34:10.551 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.547+0000 7f1515193640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15100ab630 con 0x7f151013fdb0 2024-03-13T07:34:10.551 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.547+0000 7f1515193640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:10.551 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.547+0000 7f1515193640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15100b54f0 con 0x7f15100b5110 2024-03-13T07:34:10.551 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.547+0000 7f1515193640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:10.551 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.547+0000 7f1515193640 10 monclient(hunting): authenticate will time out at 1277.956422s 2024-03-13T07:34:10.551 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.547+0000 7f150ed9d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15100b5110 0x7f151013f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:10.551 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.547+0000 7f150e59c640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f15100b22c0 0x7f1510142600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:55164/0 (socket says 172.21.15.132:55164) 2024-03-13T07:34:10.551 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.547+0000 7f150e59c640 1 -- 172.21.15.132:0/1766852383 learned_addr learned my addr 172.21.15.132:0/1766852383 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:34:10.551 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.548+0000 7f150ed9d640 10 monclient(hunting): get_auth_request con 0x7f15100b5110 auth_method 0 2024-03-13T07:34:10.552 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.548+0000 7f150ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:10.552 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.548+0000 7f150ed9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:10.552 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.548+0000 7f150ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:10.552 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.548+0000 7f150ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:10.552 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.548+0000 7f150ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:10.552 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.548+0000 7f150ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:10.553 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.550+0000 7f150ed9d640 10 monclient(hunting): handle_auth_done global_id 4196 payload 274 2024-03-13T07:34:10.553 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.550+0000 7f150ed9d640 1 -- 172.21.15.132:0/1766852383 >> v1:172.21.15.195:6789/0 conn(0x7f151013fdb0 legacy=0x7f1510142d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:10.553 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.550+0000 7f150ed9d640 1 -- 172.21.15.132:0/1766852383 >> v1:172.21.15.132:6789/0 conn(0x7f15100b22c0 legacy=0x7f1510142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:10.553 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.550+0000 7f150ed9d640 10 monclient: _finish_hunting 0 2024-03-13T07:34:10.553 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.550+0000 7f150ed9d640 1 monclient: found mon.noname-b 2024-03-13T07:34:10.553 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.550+0000 7f150ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:10.554 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.550+0000 7f150ed9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T07:34:10.554 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.550+0000 7f150ed9d640 1 -- 172.21.15.132:0/1766852383 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15100aa1a0 con 0x7f15100b5110 2024-03-13T07:34:10.554 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.553+0000 7f150ed9d640 1 --2- 172.21.15.132:0/1766852383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15100b5110 0x7f151013f870 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1500009750 tx=0x7f150002b680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=defc509992318fbc server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:10.565 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f150dd9b640 1 -- 172.21.15.132:0/1766852383 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1500037070 con 0x7f15100b5110 2024-03-13T07:34:10.565 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f150dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:10.565 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f150dd9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:10.565 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f150dd9b640 10 monclient: dump: 2024-03-13T07:34:10.565 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:10.565 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:10.565 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:10.565 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:10.565 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:10.565 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:10.565 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:10.565 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:10.565 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:10.566 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:10.566 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f150dd9b640 10 monclient: _finish_auth 0 2024-03-13T07:34:10.566 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f150dd9b640 10 monclient: _check_auth_tickets 2024-03-13T07:34:10.566 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f150dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:10.566 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f1515193640 5 monclient: authenticate success, global_id 4196 2024-03-13T07:34:10.566 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f1515193640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:10.566 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f150dd9b640 1 -- 172.21.15.132:0/1766852383 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1500030710 con 0x7f15100b5110 2024-03-13T07:34:10.566 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f150dd9b640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:10.566 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f1515193640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:10.566 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f1515193640 4 set_mon_vals no callback set 2024-03-13T07:34:10.566 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f150dd9b640 1 -- 172.21.15.132:0/1766852383 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1500037070 con 0x7f15100b5110 2024-03-13T07:34:10.566 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f150dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:10.566 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f150dd9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:10.566 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.557+0000 7f150dd9b640 10 monclient: dump: 2024-03-13T07:34:10.567 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:10.567 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:10.567 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:10.567 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:10.567 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:10.567 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:10.567 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:10.567 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:10.567 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:10.567 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:10.567 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.558+0000 7f1515193640 10 monclient: shutdown 2024-03-13T07:34:10.567 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.558+0000 7f1515193640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:10.567 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.558+0000 7f1515193640 1 -- 172.21.15.132:0/1766852383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15100b5110 msgr2=0x7f151013f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:10.567 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.558+0000 7f1515193640 1 --2- 172.21.15.132:0/1766852383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15100b5110 0x7f151013f870 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1500009750 tx=0x7f150002b680 comp rx=0 tx=0).stop 2024-03-13T07:34:10.567 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.558+0000 7f1515193640 1 -- 172.21.15.132:0/1766852383 shutdown_connections 2024-03-13T07:34:10.568 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.558+0000 7f1515193640 1 --2- 172.21.15.132:0/1766852383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15100b5110 0x7f151013f870 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1500009750 tx=0x7f150002b680 comp rx=0 tx=0).stop 2024-03-13T07:34:10.568 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.558+0000 7f1515193640 1 -- 172.21.15.132:0/1766852383 >> 172.21.15.132:0/1766852383 conn(0x7f1510072780 msgr2=0x7f1510072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:10.568 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.558+0000 7f1515193640 1 -- 172.21.15.132:0/1766852383 shutdown_connections 2024-03-13T07:34:10.568 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.558+0000 7f1515193640 1 -- 172.21.15.132:0/1766852383 wait complete. 2024-03-13T07:34:10.568 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.559+0000 7f1515193640 10 monclient: build_initial_monmap 2024-03-13T07:34:10.568 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.559+0000 7f1515193640 10 monclient: monmap: 2024-03-13T07:34:10.568 INFO:teuthology.misc.health.smithi132.stderr:epoch 0 2024-03-13T07:34:10.568 INFO:teuthology.misc.health.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:10.568 INFO:teuthology.misc.health.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:10.568 INFO:teuthology.misc.health.smithi132.stderr:created 0.000000 2024-03-13T07:34:10.568 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:10.568 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:10.568 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:10.568 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:10.569 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:10.569 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:10.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.559+0000 7f1515193640 1 librados: starting msgr at 2024-03-13T07:34:10.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.559+0000 7f1515193640 1 librados: starting objecter 2024-03-13T07:34:10.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.560+0000 7f1515193640 1 Processor -- start 2024-03-13T07:34:10.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.560+0000 7f1515193640 1 -- start start 2024-03-13T07:34:10.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.560+0000 7f1515193640 1 librados: setting wanted keys 2024-03-13T07:34:10.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.560+0000 7f1515193640 1 librados: calling monclient init 2024-03-13T07:34:10.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.560+0000 7f1515193640 10 monclient: init 2024-03-13T07:34:10.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.560+0000 7f1515193640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:10.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.560+0000 7f1515193640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T07:34:10.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.560+0000 7f1515193640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15100b22c0 0x7f151006e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:10.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.560+0000 7f1515193640 10 monclient(hunting): picked mon.noname-a con 0x7f15100b22c0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:10.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.560+0000 7f1515193640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151013fdb0 0x7f15101dc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:10.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f1515193640 10 monclient(hunting): picked mon.noname-c con 0x7f151013fdb0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:10.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f1515193640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1510146470 0x7f15101dc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:10.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f1515193640 10 monclient(hunting): picked mon.noname-b con 0x7f1510146470 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:10.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f1515193640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:10.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f1515193640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151014de60 con 0x7f15100b22c0 2024-03-13T07:34:10.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f1515193640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:10.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f1515193640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1510140240 con 0x7f1510146470 2024-03-13T07:34:10.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f1515193640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:10.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f1515193640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151014dfe0 con 0x7f151013fdb0 2024-03-13T07:34:10.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f1515193640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:10.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f1515193640 10 monclient(hunting): authenticate will time out at 1277.970309s 2024-03-13T07:34:10.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f150e59c640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151013fdb0 0x7f15101dc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:10.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f150ed9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15100b22c0 0x7f151006e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:10.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f150e59c640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151013fdb0 0x7f15101dc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:42962/0 (socket says 172.21.15.132:42962) 2024-03-13T07:34:10.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f150e59c640 1 -- 172.21.15.132:0/3900085225 learned_addr learned my addr 172.21.15.132:0/3900085225 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:10.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f150e59c640 10 monclient(hunting): get_auth_request con 0x7f151013fdb0 auth_method 0 2024-03-13T07:34:10.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f150e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:10.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f150e59c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:10.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f150e59c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:10.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f150ed9d640 10 monclient(hunting): get_auth_request con 0x7f15100b22c0 auth_method 0 2024-03-13T07:34:10.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f150ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:10.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f150ed9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:10.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.561+0000 7f150ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:10.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.562+0000 7f150f59e640 1 --2- 172.21.15.132:0/3900085225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1510146470 0x7f15101dc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:10.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.562+0000 7f150f59e640 10 monclient(hunting): get_auth_request con 0x7f1510146470 auth_method 0 2024-03-13T07:34:10.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.562+0000 7f150f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:10.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.562+0000 7f150f59e640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:10.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.562+0000 7f150f59e640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:10.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.562+0000 7f150ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:10.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.562+0000 7f150ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:10.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.562+0000 7f150ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:10.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.562+0000 7f150e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:10.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.562+0000 7f150e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:10.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.562+0000 7f150e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:10.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.562+0000 7f150f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:10.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.562+0000 7f150f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:10.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.562+0000 7f150f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:10.574 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.565+0000 7f150ed9d640 10 monclient(hunting): handle_auth_done global_id 4194 payload 824 2024-03-13T07:34:10.574 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.565+0000 7f150ed9d640 1 -- 172.21.15.132:0/3900085225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151013fdb0 msgr2=0x7f15101dc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:10.574 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.565+0000 7f150ed9d640 1 --2- 172.21.15.132:0/3900085225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151013fdb0 0x7f15101dc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:10.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.565+0000 7f150ed9d640 1 -- 172.21.15.132:0/3900085225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1510146470 msgr2=0x7f15101dc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:10.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.565+0000 7f150ed9d640 1 --2- 172.21.15.132:0/3900085225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1510146470 0x7f15101dc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:10.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.565+0000 7f150ed9d640 10 monclient: _finish_hunting 0 2024-03-13T07:34:10.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.565+0000 7f150ed9d640 1 monclient: found mon.noname-a 2024-03-13T07:34:10.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.565+0000 7f150ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:10.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.565+0000 7f150ed9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:10.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.565+0000 7f150ed9d640 1 -- 172.21.15.132:0/3900085225 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151006f880 con 0x7f15100b22c0 2024-03-13T07:34:10.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.568+0000 7f150ed9d640 1 --2- 172.21.15.132:0/3900085225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15100b22c0 0x7f151006e1c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f15100b4920 tx=0x7f15000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:10.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.568+0000 7f14f77fe640 10 client.?.objecter ms_handle_connect 0x7f15100b22c0 2024-03-13T07:34:10.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.568+0000 7f14f77fe640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:10.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f14f77fe640 1 -- 172.21.15.132:0/3900085225 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1500041020 con 0x7f15100b22c0 2024-03-13T07:34:10.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f14f77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:10.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f14f77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:10.576 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f14f77fe640 10 monclient: dump: 2024-03-13T07:34:10.576 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:10.576 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:10.576 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:10.576 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:10.576 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:10.576 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:10.576 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:10.576 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:10.576 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:10.576 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:10.576 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f14f77fe640 10 monclient: _finish_auth 0 2024-03-13T07:34:10.576 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f14f77fe640 10 monclient: _check_auth_tickets 2024-03-13T07:34:10.576 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f14f77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:10.576 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f14f77fe640 1 -- 172.21.15.132:0/3900085225 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1500030e70 con 0x7f15100b22c0 2024-03-13T07:34:10.577 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f14f77fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:10.577 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f14f77fe640 1 -- 172.21.15.132:0/3900085225 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1500037070 con 0x7f15100b22c0 2024-03-13T07:34:10.577 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f14f77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:10.577 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f14f77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:10.577 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f14f77fe640 10 monclient: dump: 2024-03-13T07:34:10.577 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:10.577 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:10.577 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:10.577 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:10.577 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:10.577 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:10.577 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:10.577 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:10.577 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:10.577 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:10.578 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f150d59a640 4 set_mon_vals no callback set 2024-03-13T07:34:10.578 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f1515193640 5 monclient: authenticate success, global_id 4194 2024-03-13T07:34:10.578 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f1515193640 10 monclient: _renew_subs 2024-03-13T07:34:10.578 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f1515193640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:10.578 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f1515193640 1 -- 172.21.15.132:0/3900085225 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151006fb60 con 0x7f15100b22c0 2024-03-13T07:34:10.578 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.572+0000 7f1515193640 10 client.4194.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:10.578 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.573+0000 7f1515193640 10 monclient: _renew_subs 2024-03-13T07:34:10.578 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.573+0000 7f1515193640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:10.578 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.573+0000 7f1515193640 1 -- 172.21.15.132:0/3900085225 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15101dcf10 con 0x7f15100b22c0 2024-03-13T07:34:10.578 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.573+0000 7f1515193640 1 librados: init done 2024-03-13T07:34:10.578 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.574+0000 7f1515193640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:10.578 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.574+0000 7f1515193640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:10.578 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.574+0000 7f1515193640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:10.578 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.574+0000 7f1515193640 1 -- 172.21.15.132:0/3900085225 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151006db90 con 0x7f15100b22c0 2024-03-13T07:34:10.604 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.603+0000 7f14f77fe640 1 -- 172.21.15.132:0/3900085225 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 4) v1 ==== 90080+0+0 (secure 0 0 0) 0x7f1500044020 con 0x7f15100b22c0 2024-03-13T07:34:10.604 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.604+0000 7f14f77fe640 1 --2- 172.21.15.132:0/3900085225 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f14fc06d4d0 0x7f14fc06f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:10.605 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.604+0000 7f150e59c640 1 --2- 172.21.15.132:0/3900085225 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f14fc06d4d0 0x7f14fc06f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:10.605 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.604+0000 7f150e59c640 10 monclient: get_auth_request con 0x7f14fc06d4d0 auth_method 0 2024-03-13T07:34:10.605 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.604+0000 7f14f77fe640 10 client.4194.objecter ms_handle_connect 0x7f14fc06d4d0 2024-03-13T07:34:10.605 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.604+0000 7f150e59c640 1 --2- 172.21.15.132:0/3900085225 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f14fc06d4d0 0x7f14fc06f990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f15101455e0 tx=0x7f1504009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:10.742 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:17.268 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:10.904 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.904+0000 7f14f77fe640 1 -- 172.21.15.132:0/3900085225 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f15000794d0 con 0x7f15100b22c0 2024-03-13T07:34:10.905 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.904+0000 7f14f77fe640 10 client.4194.objecter ms_dispatch 0x7f1510000b90 osd_map(10..10 src has 1..10) v4 2024-03-13T07:34:10.905 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.904+0000 7f14f77fe640 3 client.4194.objecter handle_osd_map got epochs [10,10] > 0 2024-03-13T07:34:10.905 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.904+0000 7f14f77fe640 3 client.4194.objecter handle_osd_map decoding full epoch 10 2024-03-13T07:34:10.905 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.904+0000 7f14f77fe640 20 client.4194.objecter dump_active .. 0 homeless 2024-03-13T07:34:10.905 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.904+0000 7f14f77fe640 1 -- 172.21.15.132:0/3900085225 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f150004b030 con 0x7f15100b22c0 2024-03-13T07:34:10.905 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.904+0000 7f14f77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:10.905 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:10.904+0000 7f14f77fe640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:11.145 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.144+0000 7f1515193640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:11.145 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.144+0000 7f1515193640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:11.145 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.144+0000 7f1515193640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:11.145 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.144+0000 7f1515193640 1 -- 172.21.15.132:0/3900085225 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1510068990 con 0x7f15100b22c0 2024-03-13T07:34:11.194 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:17.718 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:11.195 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.195+0000 7f14f77fe640 1 -- 172.21.15.132:0/3900085225 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f150007a840 con 0x7f15100b22c0 2024-03-13T07:34:11.196 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.195+0000 7f14f77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:11.196 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.195+0000 7f14f77fe640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:11.196 INFO:teuthology.misc.health.smithi132.stdout: 2024-03-13T07:34:11.196 INFO:teuthology.misc.health.smithi132.stdout:{"epoch":10,"fsid":"74e4bc2f-6a5e-4423-a8c8-807970a2f96c","created":"2024-03-13T07:33:47.494293+0000","modified":"2024-03-13T07:33:53.171597+0000","last_up_change":"0.000000","last_in_change":"2024-03-13T07:33:53.171597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0edb216c-bc1e-4865-aafe-218611f3eb09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6be0ae6b-951e-4279-ba0a-9b05b6e6e85e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"18008b7f-cfe7-4450-a3cf-aab63ebbe99d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"411b6066-4b9f-40e7-b92d-4ba914eab63e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1a719246-ae04-42b0-bd9c-fc318dc57ecb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d9ee991-241b-4507-ba2f-e3b5e2844d1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"af8b34fd-927b-4860-971b-9fddf699a0dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"19a39645-8664-463b-bae3-d7aa1026e5f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T07:34:11.199 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 10 librados: watch_flush enter 2024-03-13T07:34:11.199 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 10 librados: watch_flush exit 2024-03-13T07:34:11.199 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 20 client.4194.objecter shutdown clearing up homeless session... 2024-03-13T07:34:11.199 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 10 client.4194.objecter successfully canceled tick 2024-03-13T07:34:11.199 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 1 -- 172.21.15.132:0/3900085225 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f14fc06d4d0 msgr2=0x7f14fc06f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:11.199 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 1 --2- 172.21.15.132:0/3900085225 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f14fc06d4d0 0x7f14fc06f990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f15101455e0 tx=0x7f1504009040 comp rx=0 tx=0).stop 2024-03-13T07:34:11.199 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 10 monclient: shutdown 2024-03-13T07:34:11.199 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:11.200 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 1 -- 172.21.15.132:0/3900085225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15100b22c0 msgr2=0x7f151006e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:11.200 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 1 --2- 172.21.15.132:0/3900085225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15100b22c0 0x7f151006e1c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f15100b4920 tx=0x7f15000046c0 comp rx=0 tx=0).stop 2024-03-13T07:34:11.200 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 1 -- 172.21.15.132:0/3900085225 shutdown_connections 2024-03-13T07:34:11.200 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 1 --2- 172.21.15.132:0/3900085225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1510146470 0x7f15101dc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:11.200 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 1 --2- 172.21.15.132:0/3900085225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151013fdb0 0x7f15101dc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:11.200 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 1 --2- 172.21.15.132:0/3900085225 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f14fc06d4d0 0x7f14fc06f990 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:11.200 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 1 --2- 172.21.15.132:0/3900085225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15100b22c0 0x7f151006e1c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:11.200 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 1 -- 172.21.15.132:0/3900085225 >> 172.21.15.132:0/3900085225 conn(0x7f1510072780 msgr2=0x7f15100b5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:11.200 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 1 -- 172.21.15.132:0/3900085225 shutdown_connections 2024-03-13T07:34:11.200 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 1 -- 172.21.15.132:0/3900085225 wait complete. 2024-03-13T07:34:11.200 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:11.198+0000 7f1515193640 1 librados: shutdown 2024-03-13T07:34:11.209 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-13T07:34:11.254 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:17.793 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:11.640 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:18.168 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:11.644 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:18.187 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:14.856 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:21.386 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:14.860 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:21.390 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:15.324 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:21.850 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:15.328 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:21.854 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:15.375 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:21.924 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:15.379 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:21.928 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:15.495 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:22.044 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:15.499 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:22.048 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:16.234 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:22.758 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:16.240 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:22.764 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:16.575 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:23.118 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:16.579 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:23.122 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:16.603 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:23.142 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:16.607 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:23.146 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:16.677 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:23.205 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:16.681 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:23.209 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:18.016 DEBUG:teuthology.orchestra.run.smithi132:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-13T07:34:18.144 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.144+0000 7f4d68f43640 10 monclient: get_monmap_and_config 2024-03-13T07:34:18.144 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.144+0000 7f4d68f43640 10 monclient: build_initial_monmap 2024-03-13T07:34:18.144 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.144+0000 7f4d68f43640 10 monclient: monmap: 2024-03-13T07:34:18.144 INFO:teuthology.misc.health.smithi132.stderr:epoch 0 2024-03-13T07:34:18.144 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:18.144 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:34:18.144839+0000 2024-03-13T07:34:18.144 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:34:18.144839+0000 2024-03-13T07:34:18.144 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:18.144 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:18.144 INFO:teuthology.misc.health.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:18.144 INFO:teuthology.misc.health.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:18.144 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:18.145 INFO:teuthology.misc.health.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:18.145 INFO:teuthology.misc.health.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:18.145 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:18.146 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.144+0000 7f4d68f43640 1 Processor -- start 2024-03-13T07:34:18.147 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.144+0000 7f4d68f43640 1 -- start start 2024-03-13T07:34:18.147 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.144+0000 7f4d68f43640 10 monclient: init 2024-03-13T07:34:18.147 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.145+0000 7f4d68f43640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:18.147 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.145+0000 7f4d68f43640 10 monclient: _add_conns ranks=[2,0,1,3,4] 2024-03-13T07:34:18.147 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.145+0000 7f4d68f43640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b5180 0x7f4d640a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:18.147 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.145+0000 7f4d68f43640 10 monclient(hunting): picked mon.noname-b con 0x7f4d640b5180 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:18.147 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.146+0000 7f4d68f43640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4d640aa210 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:18.147 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.146+0000 7f4d68f43640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4d6413f490 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:18.147 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.146+0000 7f4d68f43640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d640ab350 con 0x7f4d640aa210 2024-03-13T07:34:18.147 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.146+0000 7f4d6259c640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b5180 0x7f4d640a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:18.147 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.146+0000 7f4d68f43640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d640ab630 con 0x7f4d6413f490 2024-03-13T07:34:18.147 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.146+0000 7f4d68f43640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:18.147 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.146+0000 7f4d68f43640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d640b5560 con 0x7f4d640b5180 2024-03-13T07:34:18.147 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.146+0000 7f4d68f43640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:18.148 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.146+0000 7f4d68f43640 10 monclient(hunting): authenticate will time out at 1285.554911s 2024-03-13T07:34:18.148 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.147+0000 7f4d61d9b640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f4d640aa210 0x7f4d64145ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:40172/0 (socket says 172.21.15.132:40172) 2024-03-13T07:34:18.148 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.147+0000 7f4d61d9b640 1 -- 172.21.15.132:0/2749241322 learned_addr learned my addr 172.21.15.132:0/2749241322 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:34:18.148 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.147+0000 7f4d6259c640 10 monclient(hunting): get_auth_request con 0x7f4d640b5180 auth_method 0 2024-03-13T07:34:18.148 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.147+0000 7f4d6259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:18.148 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.147+0000 7f4d6259c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:18.148 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.147+0000 7f4d6259c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:18.148 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.147+0000 7f4d6259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:18.149 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.147+0000 7f4d6259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:18.149 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.147+0000 7f4d6259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:18.150 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.150+0000 7f4d6259c640 10 monclient(hunting): handle_auth_done global_id 4202 payload 274 2024-03-13T07:34:18.150 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.150+0000 7f4d6259c640 1 -- 172.21.15.132:0/2749241322 >> v1:172.21.15.195:6789/0 conn(0x7f4d6413f490 legacy=0x7f4d6413f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:18.150 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.150+0000 7f4d6259c640 1 -- 172.21.15.132:0/2749241322 >> v1:172.21.15.132:6789/0 conn(0x7f4d640aa210 legacy=0x7f4d64145ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:18.150 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.150+0000 7f4d6259c640 10 monclient: _finish_hunting 0 2024-03-13T07:34:18.150 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.150+0000 7f4d6259c640 1 monclient: found mon.noname-b 2024-03-13T07:34:18.150 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.150+0000 7f4d6259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:18.150 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.150+0000 7f4d6259c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T07:34:18.151 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.150+0000 7f4d6259c640 1 -- 172.21.15.132:0/2749241322 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d640b47d0 con 0x7f4d640b5180 2024-03-13T07:34:18.152 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.153+0000 7f4d6259c640 1 --2- 172.21.15.132:0/2749241322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b5180 0x7f4d640a96d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4d540097d0 tx=0x7f4d5402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26663f006bcc2255 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:18.161 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d6159a640 1 -- 172.21.15.132:0/2749241322 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54035020 con 0x7f4d640b5180 2024-03-13T07:34:18.161 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d6159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:18.161 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d6159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:18.161 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d6159a640 10 monclient: dump: 2024-03-13T07:34:18.161 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:18.161 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:18.161 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:18.161 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:18.161 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:18.161 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:18.161 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:18.161 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:18.161 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:18.161 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:18.162 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d6159a640 10 monclient: _finish_auth 0 2024-03-13T07:34:18.162 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d6159a640 10 monclient: _check_auth_tickets 2024-03-13T07:34:18.162 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d6159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:18.162 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d6159a640 1 -- 172.21.15.132:0/2749241322 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d54031810 con 0x7f4d640b5180 2024-03-13T07:34:18.162 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d68f43640 5 monclient: authenticate success, global_id 4202 2024-03-13T07:34:18.162 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d68f43640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:18.162 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d6159a640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:18.162 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d68f43640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:18.162 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d68f43640 4 set_mon_vals no callback set 2024-03-13T07:34:18.162 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d6159a640 1 -- 172.21.15.132:0/2749241322 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54031be0 con 0x7f4d640b5180 2024-03-13T07:34:18.162 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d6159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:18.162 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d6159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:18.162 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.158+0000 7f4d6159a640 10 monclient: dump: 2024-03-13T07:34:18.163 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:18.163 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:18.163 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:18.163 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:18.163 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:18.163 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:18.163 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:18.163 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:18.163 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:18.163 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:18.163 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.159+0000 7f4d68f43640 10 monclient: shutdown 2024-03-13T07:34:18.163 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.159+0000 7f4d68f43640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:18.163 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.159+0000 7f4d68f43640 1 -- 172.21.15.132:0/2749241322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b5180 msgr2=0x7f4d640a96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:18.163 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.159+0000 7f4d68f43640 1 --2- 172.21.15.132:0/2749241322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b5180 0x7f4d640a96d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4d540097d0 tx=0x7f4d5402b8b0 comp rx=0 tx=0).stop 2024-03-13T07:34:18.163 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.159+0000 7f4d68f43640 1 -- 172.21.15.132:0/2749241322 shutdown_connections 2024-03-13T07:34:18.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.159+0000 7f4d68f43640 1 --2- 172.21.15.132:0/2749241322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b5180 0x7f4d640a96d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:18.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.159+0000 7f4d68f43640 1 -- 172.21.15.132:0/2749241322 >> 172.21.15.132:0/2749241322 conn(0x7f4d64072750 msgr2=0x7f4d64072b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:18.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.159+0000 7f4d68f43640 1 -- 172.21.15.132:0/2749241322 shutdown_connections 2024-03-13T07:34:18.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.159+0000 7f4d68f43640 1 -- 172.21.15.132:0/2749241322 wait complete. 2024-03-13T07:34:18.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.160+0000 7f4d68f43640 10 monclient: build_initial_monmap 2024-03-13T07:34:18.164 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.160+0000 7f4d68f43640 10 monclient: monmap: 2024-03-13T07:34:18.164 INFO:teuthology.misc.health.smithi132.stderr:epoch 0 2024-03-13T07:34:18.164 INFO:teuthology.misc.health.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:18.164 INFO:teuthology.misc.health.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:18.164 INFO:teuthology.misc.health.smithi132.stderr:created 0.000000 2024-03-13T07:34:18.164 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:18.164 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:18.165 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:18.165 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:18.165 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:18.165 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:18.165 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.160+0000 7f4d68f43640 1 librados: starting msgr at 2024-03-13T07:34:18.165 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.160+0000 7f4d68f43640 1 librados: starting objecter 2024-03-13T07:34:18.165 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.161+0000 7f4d68f43640 1 Processor -- start 2024-03-13T07:34:18.165 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.161+0000 7f4d68f43640 1 -- start start 2024-03-13T07:34:18.165 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.161+0000 7f4d68f43640 1 librados: setting wanted keys 2024-03-13T07:34:18.165 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.161+0000 7f4d68f43640 1 librados: calling monclient init 2024-03-13T07:34:18.165 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.161+0000 7f4d68f43640 10 monclient: init 2024-03-13T07:34:18.171 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.161+0000 7f4d68f43640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:18.171 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.161+0000 7f4d68f43640 10 monclient: _add_conns ranks=[1,2,0] 2024-03-13T07:34:18.171 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.161+0000 7f4d68f43640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d640aa210 0x7f4d64137e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:18.171 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.162+0000 7f4d68f43640 10 monclient(hunting): picked mon.noname-b con 0x7f4d640aa210 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:18.171 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.162+0000 7f4d68f43640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b5180 0x7f4d64142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:18.171 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.162+0000 7f4d68f43640 10 monclient(hunting): picked mon.noname-c con 0x7f4d640b5180 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:18.171 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.162+0000 7f4d68f43640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d6413f490 0x7f4d64140360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:18.171 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.162+0000 7f4d68f43640 10 monclient(hunting): picked mon.noname-a con 0x7f4d6413f490 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:18.171 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.162+0000 7f4d68f43640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:18.171 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.162+0000 7f4d68f43640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6414de80 con 0x7f4d6413f490 2024-03-13T07:34:18.171 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.162+0000 7f4d68f43640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:18.171 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.162+0000 7f4d68f43640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d640aa650 con 0x7f4d640aa210 2024-03-13T07:34:18.171 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.162+0000 7f4d68f43640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:18.172 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.162+0000 7f4d68f43640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6414e000 con 0x7f4d640b5180 2024-03-13T07:34:18.172 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.162+0000 7f4d68f43640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:18.172 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.162+0000 7f4d68f43640 10 monclient(hunting): authenticate will time out at 1285.570362s 2024-03-13T07:34:18.172 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.162+0000 7f4d61d9b640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b5180 0x7f4d64142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:18.172 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.162+0000 7f4d6259c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d640aa210 0x7f4d64137e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:18.172 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d61d9b640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b5180 0x7f4d64142f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:56246/0 (socket says 172.21.15.132:56246) 2024-03-13T07:34:18.172 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d62d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d6413f490 0x7f4d64140360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:18.172 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d61d9b640 1 -- 172.21.15.132:0/1267123903 learned_addr learned my addr 172.21.15.132:0/1267123903 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:18.172 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d61d9b640 10 monclient(hunting): get_auth_request con 0x7f4d640b5180 auth_method 0 2024-03-13T07:34:18.172 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d61d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:18.172 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d61d9b640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:18.172 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d61d9b640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:18.172 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d6259c640 10 monclient(hunting): get_auth_request con 0x7f4d640aa210 auth_method 0 2024-03-13T07:34:18.172 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d6259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:18.173 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d6259c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:18.173 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d6259c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:18.173 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d62d9d640 10 monclient(hunting): get_auth_request con 0x7f4d6413f490 auth_method 0 2024-03-13T07:34:18.173 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d62d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:18.173 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d62d9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:18.173 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d62d9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:18.173 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d61d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:18.173 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d61d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:18.173 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.163+0000 7f4d61d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:18.173 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.164+0000 7f4d6259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:18.173 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.164+0000 7f4d6259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:18.173 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.164+0000 7f4d6259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:18.173 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.164+0000 7f4d62d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:18.173 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.164+0000 7f4d62d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:18.173 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.164+0000 7f4d62d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:18.173 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.167+0000 7f4d61d9b640 10 monclient(hunting): handle_auth_done global_id 4205 payload 824 2024-03-13T07:34:18.174 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.167+0000 7f4d61d9b640 1 -- 172.21.15.132:0/1267123903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d640aa210 msgr2=0x7f4d64137e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:18.174 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.167+0000 7f4d61d9b640 1 --2- 172.21.15.132:0/1267123903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d640aa210 0x7f4d64137e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:18.174 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.167+0000 7f4d61d9b640 1 -- 172.21.15.132:0/1267123903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d6413f490 msgr2=0x7f4d64140360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:18.174 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.167+0000 7f4d61d9b640 1 --2- 172.21.15.132:0/1267123903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d6413f490 0x7f4d64140360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:18.174 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.167+0000 7f4d61d9b640 10 monclient: _finish_hunting 0 2024-03-13T07:34:18.174 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.167+0000 7f4d61d9b640 1 monclient: found mon.noname-c 2024-03-13T07:34:18.174 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.167+0000 7f4d61d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:18.174 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.167+0000 7f4d61d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:34:18.174 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.167+0000 7f4d61d9b640 1 -- 172.21.15.132:0/1267123903 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d641434c0 con 0x7f4d640b5180 2024-03-13T07:34:18.174 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.171+0000 7f4d61d9b640 1 --2- 172.21.15.132:0/1267123903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b5180 0x7f4d64142f80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800c4a0 tx=0x7f4d5800c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:18.174 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.171+0000 7f4d52ffd640 10 client.?.objecter ms_handle_connect 0x7f4d640b5180 2024-03-13T07:34:18.174 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.171+0000 7f4d52ffd640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:18.176 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.175+0000 7f4d52ffd640 1 -- 172.21.15.132:0/1267123903 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58017070 con 0x7f4d640b5180 2024-03-13T07:34:18.176 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.175+0000 7f4d52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:18.176 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.175+0000 7f4d52ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:18.176 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.175+0000 7f4d52ffd640 10 monclient: dump: 2024-03-13T07:34:18.176 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:18.176 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:18.176 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:18.176 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:18.176 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:18.176 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:18.177 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:18.177 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:18.177 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:18.177 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:18.177 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.175+0000 7f4d52ffd640 10 monclient: _finish_auth 0 2024-03-13T07:34:18.177 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.175+0000 7f4d52ffd640 10 monclient: _check_auth_tickets 2024-03-13T07:34:18.177 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.175+0000 7f4d52ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:18.177 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.175+0000 7f4d52ffd640 1 -- 172.21.15.132:0/1267123903 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d58005ce0 con 0x7f4d640b5180 2024-03-13T07:34:18.177 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.175+0000 7f4d52ffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:18.177 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.175+0000 7f4d52ffd640 1 -- 172.21.15.132:0/1267123903 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58007270 con 0x7f4d640b5180 2024-03-13T07:34:18.177 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.175+0000 7f4d52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:18.177 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.175+0000 7f4d52ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:18.177 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.175+0000 7f4d52ffd640 10 monclient: dump: 2024-03-13T07:34:18.177 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:18.178 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:18.178 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:18.178 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:18.178 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:18.178 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:18.178 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:18.178 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:18.178 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:18.178 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:18.178 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.176+0000 7f4d60d99640 4 set_mon_vals no callback set 2024-03-13T07:34:18.178 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.176+0000 7f4d68f43640 5 monclient: authenticate success, global_id 4205 2024-03-13T07:34:18.178 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.176+0000 7f4d68f43640 10 monclient: _renew_subs 2024-03-13T07:34:18.178 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.176+0000 7f4d68f43640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:18.178 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.176+0000 7f4d68f43640 1 -- 172.21.15.132:0/1267123903 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d64142aa0 con 0x7f4d640b5180 2024-03-13T07:34:18.178 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.176+0000 7f4d68f43640 10 client.4205.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:18.179 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.176+0000 7f4d68f43640 10 monclient: _renew_subs 2024-03-13T07:34:18.179 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.176+0000 7f4d68f43640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:18.179 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.176+0000 7f4d68f43640 1 -- 172.21.15.132:0/1267123903 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d641408a0 con 0x7f4d640b5180 2024-03-13T07:34:18.179 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.176+0000 7f4d68f43640 1 librados: init done 2024-03-13T07:34:18.179 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.177+0000 7f4d68f43640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:18.179 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.177+0000 7f4d68f43640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:18.180 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.177+0000 7f4d68f43640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:18.180 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.177+0000 7f4d68f43640 1 -- 172.21.15.132:0/1267123903 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6406a510 con 0x7f4d640b5180 2024-03-13T07:34:18.338 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.339+0000 7f4d52ffd640 1 -- 172.21.15.132:0/1267123903 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 4) v1 ==== 90080+0+0 (secure 0 0 0) 0x7f4d5801a070 con 0x7f4d640b5180 2024-03-13T07:34:18.339 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.339+0000 7f4d52ffd640 1 --2- 172.21.15.132:0/1267123903 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4d4406d520 0x7f4d4406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:18.339 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.339+0000 7f4d6259c640 1 --2- 172.21.15.132:0/1267123903 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4d4406d520 0x7f4d4406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:18.339 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.339+0000 7f4d6259c640 10 monclient: get_auth_request con 0x7f4d4406d520 auth_method 0 2024-03-13T07:34:18.340 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.340+0000 7f4d6259c640 1 --2- 172.21.15.132:0/1267123903 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4d4406d520 0x7f4d4406f9e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4d54036000 tx=0x7f4d54008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:18.340 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.340+0000 7f4d52ffd640 10 client.4205.objecter ms_handle_connect 0x7f4d4406d520 2024-03-13T07:34:18.500 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.500+0000 7f4d52ffd640 1 -- 172.21.15.132:0/1267123903 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4d580572b0 con 0x7f4d640b5180 2024-03-13T07:34:18.500 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.500+0000 7f4d52ffd640 10 client.4205.objecter ms_dispatch 0x7f4d64000b90 osd_map(10..10 src has 1..10) v4 2024-03-13T07:34:18.500 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.500+0000 7f4d52ffd640 3 client.4205.objecter handle_osd_map got epochs [10,10] > 0 2024-03-13T07:34:18.500 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.500+0000 7f4d52ffd640 3 client.4205.objecter handle_osd_map decoding full epoch 10 2024-03-13T07:34:18.500 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.500+0000 7f4d52ffd640 20 client.4205.objecter dump_active .. 0 homeless 2024-03-13T07:34:18.520 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.520+0000 7f4d52ffd640 1 -- 172.21.15.132:0/1267123903 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f4d58058620 con 0x7f4d640b5180 2024-03-13T07:34:18.521 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.520+0000 7f4d52ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:18.521 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.520+0000 7f4d52ffd640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:18.841 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.841+0000 7f4d68f43640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:18.841 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.841+0000 7f4d68f43640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:18.842 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.841+0000 7f4d68f43640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:18.842 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.841+0000 7f4d68f43640 1 -- 172.21.15.132:0/1267123903 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d64067ca0 con 0x7f4d640b5180 2024-03-13T07:34:18.857 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.857+0000 7f4d52ffd640 1 -- 172.21.15.132:0/1267123903 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f4d5805c6c0 con 0x7f4d640b5180 2024-03-13T07:34:18.857 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.857+0000 7f4d52ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:18.857 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.857+0000 7f4d52ffd640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:18.857 INFO:teuthology.misc.health.smithi132.stdout: 2024-03-13T07:34:18.858 INFO:teuthology.misc.health.smithi132.stdout:{"epoch":10,"fsid":"74e4bc2f-6a5e-4423-a8c8-807970a2f96c","created":"2024-03-13T07:33:47.494293+0000","modified":"2024-03-13T07:33:53.171597+0000","last_up_change":"0.000000","last_in_change":"2024-03-13T07:33:53.171597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0edb216c-bc1e-4865-aafe-218611f3eb09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6be0ae6b-951e-4279-ba0a-9b05b6e6e85e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"18008b7f-cfe7-4450-a3cf-aab63ebbe99d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"411b6066-4b9f-40e7-b92d-4ba914eab63e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1a719246-ae04-42b0-bd9c-fc318dc57ecb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d9ee991-241b-4507-ba2f-e3b5e2844d1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"af8b34fd-927b-4860-971b-9fddf699a0dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"19a39645-8664-463b-bae3-d7aa1026e5f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T07:34:18.860 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 10 librados: watch_flush enter 2024-03-13T07:34:18.860 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 10 librados: watch_flush exit 2024-03-13T07:34:18.860 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 20 client.4205.objecter shutdown clearing up homeless session... 2024-03-13T07:34:18.860 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 10 client.4205.objecter successfully canceled tick 2024-03-13T07:34:18.860 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 1 -- 172.21.15.132:0/1267123903 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4d4406d520 msgr2=0x7f4d4406f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:18.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 1 --2- 172.21.15.132:0/1267123903 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4d4406d520 0x7f4d4406f9e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4d54036000 tx=0x7f4d54008030 comp rx=0 tx=0).stop 2024-03-13T07:34:18.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 10 monclient: shutdown 2024-03-13T07:34:18.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:18.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 1 -- 172.21.15.132:0/1267123903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b5180 msgr2=0x7f4d64142f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:18.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 1 --2- 172.21.15.132:0/1267123903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b5180 0x7f4d64142f80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800c4a0 tx=0x7f4d5800c960 comp rx=0 tx=0).stop 2024-03-13T07:34:18.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 1 -- 172.21.15.132:0/1267123903 shutdown_connections 2024-03-13T07:34:18.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 1 --2- 172.21.15.132:0/1267123903 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4d4406d520 0x7f4d4406f9e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:18.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 1 --2- 172.21.15.132:0/1267123903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d6413f490 0x7f4d64140360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:18.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 1 --2- 172.21.15.132:0/1267123903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b5180 0x7f4d64142f80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:18.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 1 --2- 172.21.15.132:0/1267123903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d640aa210 0x7f4d64137e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:18.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 1 -- 172.21.15.132:0/1267123903 >> 172.21.15.132:0/1267123903 conn(0x7f4d64072750 msgr2=0x7f4d640b6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:18.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 1 -- 172.21.15.132:0/1267123903 shutdown_connections 2024-03-13T07:34:18.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.860+0000 7f4d68f43640 1 -- 172.21.15.132:0/1267123903 wait complete. 2024-03-13T07:34:18.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:18.861+0000 7f4d68f43640 1 librados: shutdown 2024-03-13T07:34:18.874 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-13T07:34:19.149 INFO:tasks.ceph.osd.6.smithi195.stderr:==00:00:00:25.679 35251== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:19.377 INFO:tasks.ceph.osd.5.smithi195.stderr:==00:00:00:25.925 35250== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:19.378 INFO:tasks.ceph.osd.7.smithi195.stderr:==00:00:00:25.904 35252== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:19.484 INFO:tasks.ceph.osd.4.smithi195.stderr:==00:00:00:26.033 35249== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:20.555 INFO:tasks.ceph.osd.3.smithi132.stderr:==00:00:00:27.078 35611== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:20.911 INFO:tasks.ceph.osd.1.smithi132.stderr:==00:00:00:27.450 35595== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:21.279 INFO:tasks.ceph.osd.0.smithi132.stderr:==00:00:00:27.822 35594== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:21.464 INFO:tasks.ceph.osd.2.smithi132.stderr:==00:00:00:27.992 35610== Warning: unimplemented fcntl command: 1038 2024-03-13T07:34:25.681 DEBUG:teuthology.orchestra.run.smithi132:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-13T07:34:25.826 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.826+0000 7f36134d0640 10 monclient: get_monmap_and_config 2024-03-13T07:34:25.826 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.826+0000 7f36134d0640 10 monclient: build_initial_monmap 2024-03-13T07:34:25.826 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.826+0000 7f36134d0640 10 monclient: monmap: 2024-03-13T07:34:25.826 INFO:teuthology.misc.health.smithi132.stderr:epoch 0 2024-03-13T07:34:25.826 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:25.826 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:34:25.827257+0000 2024-03-13T07:34:25.826 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:34:25.827257+0000 2024-03-13T07:34:25.827 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:25.827 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:25.827 INFO:teuthology.misc.health.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:25.827 INFO:teuthology.misc.health.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:25.827 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:25.827 INFO:teuthology.misc.health.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:25.827 INFO:teuthology.misc.health.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:25.827 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:25.828 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.827+0000 7f36134d0640 1 Processor -- start 2024-03-13T07:34:25.829 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.827+0000 7f36134d0640 1 -- start start 2024-03-13T07:34:25.829 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.827+0000 7f36134d0640 10 monclient: init 2024-03-13T07:34:25.829 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.828+0000 7f36134d0640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:25.829 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.828+0000 7f36134d0640 10 monclient: _add_conns ranks=[3,0,4,2,1] 2024-03-13T07:34:25.829 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.828+0000 7f36134d0640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f360c0b5180 0x7f360c0a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:25.829 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.828+0000 7f36134d0640 10 monclient(hunting): picked mon.noname-a con 0x7f360c0b5180 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:25.829 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.828+0000 7f36134d0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f360c0aa210 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:25.829 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.828+0000 7f36134d0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f360c13f490 0x7f360c13f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:25.829 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.828+0000 7f36134d0640 10 monclient(hunting): picked mon.noname-c con 0x7f360c13f490 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:25.829 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.829+0000 7f36134d0640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360c0ab350 con 0x7f360c0aa210 2024-03-13T07:34:25.829 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.829+0000 7f36134d0640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:25.829 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.829+0000 7f36134d0640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360c0b5560 con 0x7f360c0b5180 2024-03-13T07:34:25.829 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.829+0000 7f36134d0640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:25.829 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.829+0000 7f36134d0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360c0ab630 con 0x7f360c13f490 2024-03-13T07:34:25.830 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.829+0000 7f36134d0640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:25.830 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.829+0000 7f36134d0640 10 monclient(hunting): authenticate will time out at 1293.237201s 2024-03-13T07:34:25.830 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.829+0000 7f361126c640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f360c0b5180 0x7f360c0a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:25.830 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.829+0000 7f3611a6d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f360c13f490 0x7f360c13f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:25.831 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.829+0000 7f361126c640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f360c0b5180 0x7f360c0a96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35532/0 (socket says 172.21.15.132:35532) 2024-03-13T07:34:25.831 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.829+0000 7f361126c640 1 -- 172.21.15.132:0/1572804732 learned_addr learned my addr 172.21.15.132:0/1572804732 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:25.831 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.829+0000 7f361126c640 10 monclient(hunting): get_auth_request con 0x7f360c0b5180 auth_method 0 2024-03-13T07:34:25.831 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.829+0000 7f361126c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:25.831 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.829+0000 7f361126c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:25.832 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.829+0000 7f361126c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:25.832 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.830+0000 7f3611a6d640 10 monclient(hunting): get_auth_request con 0x7f360c13f490 auth_method 0 2024-03-13T07:34:25.832 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.830+0000 7f3611a6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:25.832 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.830+0000 7f3611a6d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:25.832 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.830+0000 7f3611a6d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:25.832 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.830+0000 7f3611a6d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:25.832 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.830+0000 7f3611a6d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:25.832 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.830+0000 7f3611a6d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:25.832 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.831+0000 7f361126c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:25.832 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.831+0000 7f361126c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:25.832 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.831+0000 7f361126c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:25.833 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.833+0000 7f3611a6d640 10 monclient(hunting): handle_auth_done global_id 4207 payload 274 2024-03-13T07:34:25.833 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.833+0000 7f3611a6d640 1 -- 172.21.15.132:0/1572804732 >> v2:172.21.15.132:3300/0 conn(0x7f360c0b5180 msgr2=0x7f360c0a96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:25.833 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.833+0000 7f3611a6d640 1 --2- 172.21.15.132:0/1572804732 >> v2:172.21.15.132:3300/0 conn(0x7f360c0b5180 0x7f360c0a96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:25.834 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.833+0000 7f3611a6d640 1 -- 172.21.15.132:0/1572804732 >> v1:172.21.15.132:6789/0 conn(0x7f360c0aa210 legacy=0x7f360c145ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:25.834 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.833+0000 7f3611a6d640 10 monclient: _finish_hunting 0 2024-03-13T07:34:25.834 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.833+0000 7f3611a6d640 1 monclient: found mon.noname-c 2024-03-13T07:34:25.834 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.833+0000 7f3611a6d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:25.834 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.833+0000 7f3611a6d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T07:34:25.834 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.833+0000 7f3611a6d640 1 -- 172.21.15.132:0/1572804732 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c0b47d0 con 0x7f360c13f490 2024-03-13T07:34:25.836 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.836+0000 7f3611a6d640 1 --2- 172.21.15.132:0/1572804732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f360c13f490 0x7f360c13f900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f360c0aaf70 tx=0x7f360402be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19edbdfd26f20007 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:25.843 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.840+0000 7f3603fff640 1 -- 172.21.15.132:0/1572804732 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604039070 con 0x7f360c13f490 2024-03-13T07:34:25.843 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.840+0000 7f3603fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:25.843 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.840+0000 7f3603fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:25.843 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.840+0000 7f3603fff640 10 monclient: dump: 2024-03-13T07:34:25.843 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:25.844 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:25.844 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:25.844 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:25.844 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:25.844 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:25.844 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:25.844 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:25.844 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:25.844 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:25.844 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.840+0000 7f3603fff640 10 monclient: _finish_auth 0 2024-03-13T07:34:25.844 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.840+0000 7f3603fff640 10 monclient: _check_auth_tickets 2024-03-13T07:34:25.844 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.840+0000 7f3603fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:25.844 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.840+0000 7f3603fff640 1 -- 172.21.15.132:0/1572804732 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360402cbf0 con 0x7f360c13f490 2024-03-13T07:34:25.844 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.840+0000 7f36134d0640 5 monclient: authenticate success, global_id 4207 2024-03-13T07:34:25.844 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.840+0000 7f36134d0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:25.845 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.840+0000 7f3603fff640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:25.845 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.840+0000 7f36134d0640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:25.845 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.840+0000 7f3603fff640 1 -- 172.21.15.132:0/1572804732 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604034440 con 0x7f360c13f490 2024-03-13T07:34:25.845 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.840+0000 7f36134d0640 4 set_mon_vals no callback set 2024-03-13T07:34:25.845 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.840+0000 7f3603fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:25.845 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.840+0000 7f3603fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:25.845 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.841+0000 7f3603fff640 10 monclient: dump: 2024-03-13T07:34:25.845 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:25.845 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:25.845 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:25.845 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:25.845 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:25.845 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:25.845 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:25.845 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:25.846 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:25.846 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:25.846 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.841+0000 7f36134d0640 10 monclient: shutdown 2024-03-13T07:34:25.846 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.841+0000 7f36134d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:25.846 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.841+0000 7f36134d0640 1 -- 172.21.15.132:0/1572804732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f360c13f490 msgr2=0x7f360c13f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:25.846 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.841+0000 7f36134d0640 1 --2- 172.21.15.132:0/1572804732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f360c13f490 0x7f360c13f900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f360c0aaf70 tx=0x7f360402be40 comp rx=0 tx=0).stop 2024-03-13T07:34:25.846 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.842+0000 7f36134d0640 1 -- 172.21.15.132:0/1572804732 shutdown_connections 2024-03-13T07:34:25.846 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.842+0000 7f36134d0640 1 --2- 172.21.15.132:0/1572804732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f360c13f490 0x7f360c13f900 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:25.846 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.842+0000 7f36134d0640 1 --2- 172.21.15.132:0/1572804732 >> v2:172.21.15.132:3300/0 conn(0x7f360c0b5180 0x7f360c0a96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:25.846 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.842+0000 7f36134d0640 1 -- 172.21.15.132:0/1572804732 >> 172.21.15.132:0/1572804732 conn(0x7f360c072750 msgr2=0x7f360c072b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:25.846 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.842+0000 7f36134d0640 1 -- 172.21.15.132:0/1572804732 shutdown_connections 2024-03-13T07:34:25.846 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.842+0000 7f36134d0640 1 -- 172.21.15.132:0/1572804732 wait complete. 2024-03-13T07:34:25.846 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.843+0000 7f36134d0640 10 monclient: build_initial_monmap 2024-03-13T07:34:25.846 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.843+0000 7f36134d0640 10 monclient: monmap: 2024-03-13T07:34:25.846 INFO:teuthology.misc.health.smithi132.stderr:epoch 0 2024-03-13T07:34:25.847 INFO:teuthology.misc.health.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:25.847 INFO:teuthology.misc.health.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:25.847 INFO:teuthology.misc.health.smithi132.stderr:created 0.000000 2024-03-13T07:34:25.847 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:25.847 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:25.847 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:25.847 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:25.847 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:25.847 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:25.847 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.843+0000 7f36134d0640 1 librados: starting msgr at 2024-03-13T07:34:25.847 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.843+0000 7f36134d0640 1 librados: starting objecter 2024-03-13T07:34:25.847 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.843+0000 7f36134d0640 1 Processor -- start 2024-03-13T07:34:25.847 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.843+0000 7f36134d0640 1 -- start start 2024-03-13T07:34:25.847 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.843+0000 7f36134d0640 1 librados: setting wanted keys 2024-03-13T07:34:25.847 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.843+0000 7f36134d0640 1 librados: calling monclient init 2024-03-13T07:34:25.848 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.843+0000 7f36134d0640 10 monclient: init 2024-03-13T07:34:25.852 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.844+0000 7f36134d0640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:25.852 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.844+0000 7f36134d0640 10 monclient: _add_conns ranks=[0,1,2] 2024-03-13T07:34:25.852 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.844+0000 7f36134d0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f360c0aa210 0x7f360c071af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:25.853 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.844+0000 7f36134d0640 10 monclient(hunting): picked mon.noname-a con 0x7f360c0aa210 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:25.853 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.844+0000 7f36134d0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f360c0b5180 0x7f360c142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:25.853 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.844+0000 7f36134d0640 10 monclient(hunting): picked mon.noname-b con 0x7f360c0b5180 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:25.853 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.844+0000 7f36134d0640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f360c13f490 0x7f360c1db9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:25.853 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.844+0000 7f36134d0640 10 monclient(hunting): picked mon.noname-c con 0x7f360c13f490 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:25.853 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.844+0000 7f36134d0640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:25.853 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.844+0000 7f36134d0640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c13fe40 con 0x7f360c0aa210 2024-03-13T07:34:25.853 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.844+0000 7f36134d0640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:25.853 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.844+0000 7f36134d0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c1461f0 con 0x7f360c0b5180 2024-03-13T07:34:25.853 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.844+0000 7f36134d0640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:25.853 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.844+0000 7f36134d0640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360c14ce70 con 0x7f360c13f490 2024-03-13T07:34:25.853 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.844+0000 7f36134d0640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:25.853 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.844+0000 7f36134d0640 10 monclient(hunting): authenticate will time out at 1293.252787s 2024-03-13T07:34:25.853 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.845+0000 7f361126c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f360c0aa210 0x7f360c071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:25.853 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.845+0000 7f3611a6d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f360c13f490 0x7f360c1db9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:25.854 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.845+0000 7f3611a6d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f360c13f490 0x7f360c1db9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:41110/0 (socket says 172.21.15.132:41110) 2024-03-13T07:34:25.854 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.845+0000 7f3611a6d640 1 -- 172.21.15.132:0/2063891441 learned_addr learned my addr 172.21.15.132:0/2063891441 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:25.854 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.845+0000 7f3611a6d640 10 monclient(hunting): get_auth_request con 0x7f360c13f490 auth_method 0 2024-03-13T07:34:25.854 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.845+0000 7f3611a6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:25.854 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.845+0000 7f3611a6d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:25.854 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.845+0000 7f3611a6d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:25.854 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.845+0000 7f361126c640 10 monclient(hunting): get_auth_request con 0x7f360c0aa210 auth_method 0 2024-03-13T07:34:25.854 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.845+0000 7f361126c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:25.854 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.845+0000 7f361126c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:25.854 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.845+0000 7f361126c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:25.854 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.846+0000 7f3610a6b640 1 --2- 172.21.15.132:0/2063891441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f360c0b5180 0x7f360c142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:25.854 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.846+0000 7f3610a6b640 10 monclient(hunting): get_auth_request con 0x7f360c0b5180 auth_method 0 2024-03-13T07:34:25.854 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.846+0000 7f3610a6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:25.854 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.846+0000 7f3610a6b640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:25.855 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.846+0000 7f3610a6b640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:25.855 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.846+0000 7f3611a6d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:25.855 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.846+0000 7f3611a6d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:25.855 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.846+0000 7f3611a6d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:25.855 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.846+0000 7f361126c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:25.855 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.846+0000 7f361126c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:25.855 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.846+0000 7f361126c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:25.855 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.847+0000 7f3610a6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:25.855 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.847+0000 7f3610a6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:25.855 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.847+0000 7f3610a6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:25.855 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.850+0000 7f3611a6d640 10 monclient(hunting): handle_auth_done global_id 4208 payload 824 2024-03-13T07:34:25.855 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.850+0000 7f3611a6d640 1 -- 172.21.15.132:0/2063891441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f360c0b5180 msgr2=0x7f360c142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:25.855 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.850+0000 7f3611a6d640 1 --2- 172.21.15.132:0/2063891441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f360c0b5180 0x7f360c142cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:25.855 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.850+0000 7f3611a6d640 1 -- 172.21.15.132:0/2063891441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f360c0aa210 msgr2=0x7f360c071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:25.855 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.850+0000 7f3611a6d640 1 --2- 172.21.15.132:0/2063891441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f360c0aa210 0x7f360c071af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:25.856 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.850+0000 7f3611a6d640 10 monclient: _finish_hunting 0 2024-03-13T07:34:25.856 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.850+0000 7f3611a6d640 1 monclient: found mon.noname-c 2024-03-13T07:34:25.856 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.850+0000 7f3611a6d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:25.856 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.850+0000 7f3611a6d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:34:25.856 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.850+0000 7f3611a6d640 1 -- 172.21.15.132:0/2063891441 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c143420 con 0x7f360c13f490 2024-03-13T07:34:25.856 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.854+0000 7f3611a6d640 1 --2- 172.21.15.132:0/2063891441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f360c13f490 0x7f360c1db9c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3604009e80 tx=0x7f3604004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:25.856 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.854+0000 7f3601ffb640 10 client.?.objecter ms_handle_connect 0x7f360c13f490 2024-03-13T07:34:25.857 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.854+0000 7f3601ffb640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:25.860 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f3601ffb640 1 -- 172.21.15.132:0/2063891441 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604039050 con 0x7f360c13f490 2024-03-13T07:34:25.860 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f3601ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:25.860 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f3601ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:25.860 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f3601ffb640 10 monclient: dump: 2024-03-13T07:34:25.861 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:25.861 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:25.861 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:25.861 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:25.861 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:25.861 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:25.861 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:25.861 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:25.861 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:25.861 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:25.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f3601ffb640 10 monclient: _finish_auth 0 2024-03-13T07:34:25.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f3601ffb640 10 monclient: _check_auth_tickets 2024-03-13T07:34:25.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f3601ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:25.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f3601ffb640 1 -- 172.21.15.132:0/2063891441 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3604045070 con 0x7f360c13f490 2024-03-13T07:34:25.861 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f36134d0640 5 monclient: authenticate success, global_id 4208 2024-03-13T07:34:25.862 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f36134d0640 10 monclient: _renew_subs 2024-03-13T07:34:25.862 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f36134d0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:25.862 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f36134d0640 1 -- 172.21.15.132:0/2063891441 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c142850 con 0x7f360c13f490 2024-03-13T07:34:25.862 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f3601ffb640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:25.862 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f3601ffb640 1 -- 172.21.15.132:0/2063891441 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604009450 con 0x7f360c13f490 2024-03-13T07:34:25.862 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f3601ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:25.863 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f3601ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:25.863 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f3601ffb640 10 monclient: dump: 2024-03-13T07:34:25.863 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:25.863 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:25.863 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:25.863 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:25.863 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:25.863 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:25.863 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:25.863 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:25.863 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:25.863 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:25.863 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f36134d0640 10 client.4208.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:25.863 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f36134d0640 10 monclient: _renew_subs 2024-03-13T07:34:25.863 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f36134d0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:25.864 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f36134d0640 1 -- 172.21.15.132:0/2063891441 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360c1dc0e0 con 0x7f360c13f490 2024-03-13T07:34:25.864 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f36134d0640 1 librados: init done 2024-03-13T07:34:25.864 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.860+0000 7f36037fe640 4 set_mon_vals no callback set 2024-03-13T07:34:25.864 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.862+0000 7f36134d0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:25.864 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.862+0000 7f36134d0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:25.864 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.862+0000 7f36134d0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:25.864 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.862+0000 7f36134d0640 1 -- 172.21.15.132:0/2063891441 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360c06a510 con 0x7f360c13f490 2024-03-13T07:34:25.895 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.895+0000 7f3601ffb640 1 -- 172.21.15.132:0/2063891441 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 4) v1 ==== 90080+0+0 (secure 0 0 0) 0x7f360403e850 con 0x7f360c13f490 2024-03-13T07:34:25.895 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.895+0000 7f3601ffb640 1 --2- 172.21.15.132:0/2063891441 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f35f006d520 0x7f35f006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:25.895 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.895+0000 7f361126c640 1 --2- 172.21.15.132:0/2063891441 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f35f006d520 0x7f35f006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:25.895 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.895+0000 7f361126c640 10 monclient: get_auth_request con 0x7f35f006d520 auth_method 0 2024-03-13T07:34:25.895 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.896+0000 7f3601ffb640 10 client.4208.objecter ms_handle_connect 0x7f35f006d520 2024-03-13T07:34:25.896 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:25.896+0000 7f361126c640 1 --2- 172.21.15.132:0/2063891441 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f35f006d520 0x7f35f006f9e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3608009bd0 tx=0x7f3608009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:26.335 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.335+0000 7f3601ffb640 1 -- 172.21.15.132:0/2063891441 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f360407b2b0 con 0x7f360c13f490 2024-03-13T07:34:26.335 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.335+0000 7f3601ffb640 10 client.4208.objecter ms_dispatch 0x7f360c000b90 osd_map(10..10 src has 1..10) v4 2024-03-13T07:34:26.335 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.335+0000 7f3601ffb640 3 client.4208.objecter handle_osd_map got epochs [10,10] > 0 2024-03-13T07:34:26.335 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.335+0000 7f3601ffb640 3 client.4208.objecter handle_osd_map decoding full epoch 10 2024-03-13T07:34:26.335 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.335+0000 7f3601ffb640 20 client.4208.objecter dump_active .. 0 homeless 2024-03-13T07:34:26.335 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.335+0000 7f3601ffb640 1 -- 172.21.15.132:0/2063891441 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f360407c620 con 0x7f360c13f490 2024-03-13T07:34:26.335 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.335+0000 7f3601ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:26.335 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.335+0000 7f3601ffb640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:26.637 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.637+0000 7f36134d0640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:26.637 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.637+0000 7f36134d0640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:26.637 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.637+0000 7f36134d0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:26.637 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.637+0000 7f36134d0640 1 -- 172.21.15.132:0/2063891441 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f360c1435f0 con 0x7f360c13f490 2024-03-13T07:34:26.653 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.653+0000 7f3601ffb640 1 -- 172.21.15.132:0/2063891441 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f36040806c0 con 0x7f360c13f490 2024-03-13T07:34:26.653 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.653+0000 7f3601ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:26.653 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.653+0000 7f3601ffb640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:26.653 INFO:teuthology.misc.health.smithi132.stdout: 2024-03-13T07:34:26.653 INFO:teuthology.misc.health.smithi132.stdout:{"epoch":10,"fsid":"74e4bc2f-6a5e-4423-a8c8-807970a2f96c","created":"2024-03-13T07:33:47.494293+0000","modified":"2024-03-13T07:33:53.171597+0000","last_up_change":"0.000000","last_in_change":"2024-03-13T07:33:53.171597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0edb216c-bc1e-4865-aafe-218611f3eb09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6be0ae6b-951e-4279-ba0a-9b05b6e6e85e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"18008b7f-cfe7-4450-a3cf-aab63ebbe99d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"411b6066-4b9f-40e7-b92d-4ba914eab63e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1a719246-ae04-42b0-bd9c-fc318dc57ecb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d9ee991-241b-4507-ba2f-e3b5e2844d1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"af8b34fd-927b-4860-971b-9fddf699a0dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"19a39645-8664-463b-bae3-d7aa1026e5f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T07:34:26.655 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 10 librados: watch_flush enter 2024-03-13T07:34:26.655 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 10 librados: watch_flush exit 2024-03-13T07:34:26.655 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 20 client.4208.objecter shutdown clearing up homeless session... 2024-03-13T07:34:26.655 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 10 client.4208.objecter successfully canceled tick 2024-03-13T07:34:26.655 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 1 -- 172.21.15.132:0/2063891441 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f35f006d520 msgr2=0x7f35f006f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:26.655 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 1 --2- 172.21.15.132:0/2063891441 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f35f006d520 0x7f35f006f9e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3608009bd0 tx=0x7f3608009240 comp rx=0 tx=0).stop 2024-03-13T07:34:26.655 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 10 monclient: shutdown 2024-03-13T07:34:26.655 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:26.656 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 1 -- 172.21.15.132:0/2063891441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f360c13f490 msgr2=0x7f360c1db9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:26.656 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 1 --2- 172.21.15.132:0/2063891441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f360c13f490 0x7f360c1db9c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3604009e80 tx=0x7f3604004a40 comp rx=0 tx=0).stop 2024-03-13T07:34:26.656 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 1 -- 172.21.15.132:0/2063891441 shutdown_connections 2024-03-13T07:34:26.656 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 1 --2- 172.21.15.132:0/2063891441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f360c13f490 0x7f360c1db9c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:26.656 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 1 --2- 172.21.15.132:0/2063891441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f360c0b5180 0x7f360c142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:26.656 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 1 --2- 172.21.15.132:0/2063891441 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f35f006d520 0x7f35f006f9e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:26.656 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 1 --2- 172.21.15.132:0/2063891441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f360c0aa210 0x7f360c071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:26.656 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 1 -- 172.21.15.132:0/2063891441 >> 172.21.15.132:0/2063891441 conn(0x7f360c072750 msgr2=0x7f360c0b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:26.656 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 1 -- 172.21.15.132:0/2063891441 shutdown_connections 2024-03-13T07:34:26.657 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 1 -- 172.21.15.132:0/2063891441 wait complete. 2024-03-13T07:34:26.657 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:26.655+0000 7f36134d0640 1 librados: shutdown 2024-03-13T07:34:26.668 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-13T07:34:27.853 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:34:27.850+0000 5dc3480 -1 osd.7 0 log_to_monitors true 2024-03-13T07:34:28.147 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T07:34:28.143+0000 5dc3480 -1 osd.4 0 log_to_monitors true 2024-03-13T07:34:28.212 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-03-13T07:34:28.207+0000 5dc3480 -1 osd.5 0 log_to_monitors true 2024-03-13T07:34:28.350 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-03-13T07:34:28.345+0000 5dc3480 -1 osd.6 0 log_to_monitors true 2024-03-13T07:34:29.566 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-03-13T07:34:29.555+0000 5dc3480 -1 osd.3 0 log_to_monitors true 2024-03-13T07:34:29.692 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:29.689+0000 5dc3480 -1 osd.1 0 log_to_monitors true 2024-03-13T07:34:30.317 INFO:tasks.ceph.osd.0.smithi132.stderr:2024-03-13T07:34:30.223+0000 5dc3480 -1 osd.0 0 log_to_monitors true 2024-03-13T07:34:30.404 INFO:tasks.ceph.osd.2.smithi132.stderr:2024-03-13T07:34:30.402+0000 5dc3480 -1 osd.2 0 log_to_monitors true 2024-03-13T07:34:31.104 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-03-13T07:34:31.101+0000 b47a640 -1 osd.5 0 waiting for initial osdmap 2024-03-13T07:34:31.185 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:34:31.182+0000 bc8e640 -1 osd.7 0 waiting for initial osdmap 2024-03-13T07:34:31.542 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-03-13T07:34:31.527+0000 10255640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T07:34:31.587 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:34:31.573+0000 10255640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T07:34:32.381 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-03-13T07:34:32.376+0000 b47a640 -1 osd.6 0 waiting for initial osdmap 2024-03-13T07:34:32.806 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-03-13T07:34:32.792+0000 10255640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T07:34:33.475 DEBUG:teuthology.orchestra.run.smithi132:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-13T07:34:33.565 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.565+0000 7fb6081d4640 10 monclient: get_monmap_and_config 2024-03-13T07:34:33.565 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.565+0000 7fb6081d4640 10 monclient: build_initial_monmap 2024-03-13T07:34:33.565 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.565+0000 7fb6081d4640 10 monclient: monmap: 2024-03-13T07:34:33.565 INFO:teuthology.misc.health.smithi132.stderr:epoch 0 2024-03-13T07:34:33.565 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:33.565 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:34:33.565981+0000 2024-03-13T07:34:33.565 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:34:33.565981+0000 2024-03-13T07:34:33.565 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:33.565 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:33.566 INFO:teuthology.misc.health.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:33.566 INFO:teuthology.misc.health.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:33.566 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:33.566 INFO:teuthology.misc.health.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:33.566 INFO:teuthology.misc.health.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:33.566 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:33.566 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.565+0000 7fb6081d4640 1 Processor -- start 2024-03-13T07:34:33.568 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.566+0000 7fb6081d4640 1 -- start start 2024-03-13T07:34:33.568 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.566+0000 7fb6081d4640 10 monclient: init 2024-03-13T07:34:33.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.566+0000 7fb6081d4640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:33.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.566+0000 7fb6081d4640 10 monclient: _add_conns ranks=[1,3,4,2,0] 2024-03-13T07:34:33.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.566+0000 7fb6081d4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb6000b5180 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:33.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.566+0000 7fb6081d4640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb6000a96d0 0x7fb6000aa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:33.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.566+0000 7fb6081d4640 10 monclient(hunting): picked mon.noname-a con 0x7fb6000a96d0 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:33.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.566+0000 7fb6081d4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb60013f490 0x7fb60013f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:33.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.566+0000 7fb6081d4640 10 monclient(hunting): picked mon.noname-c con 0x7fb60013f490 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:33.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.566+0000 7fb6081d4640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6000ab350 con 0x7fb6000b5180 2024-03-13T07:34:33.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.566+0000 7fb6081d4640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:33.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.566+0000 7fb6081d4640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6000b5560 con 0x7fb6000a96d0 2024-03-13T07:34:33.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.566+0000 7fb6081d4640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:33.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.566+0000 7fb6081d4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6000ab630 con 0x7fb60013f490 2024-03-13T07:34:33.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.566+0000 7fb6081d4640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:33.569 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.566+0000 7fb6081d4640 10 monclient(hunting): authenticate will time out at 1300.974921s 2024-03-13T07:34:33.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.567+0000 7fb606771640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb60013f490 0x7fb60013f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:33.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.567+0000 7fb60576f640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb6000a96d0 0x7fb6000aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:33.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.567+0000 7fb605f70640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fb6000b5180 0x7fb600145ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.132:40388/0 (socket says 172.21.15.132:40388) 2024-03-13T07:34:33.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.567+0000 7fb605f70640 1 -- 172.21.15.132:0/348854733 learned_addr learned my addr 172.21.15.132:0/348854733 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:34:33.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.567+0000 7fb60576f640 10 monclient(hunting): get_auth_request con 0x7fb6000a96d0 auth_method 0 2024-03-13T07:34:33.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.567+0000 7fb60576f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:33.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.567+0000 7fb60576f640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:33.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.567+0000 7fb60576f640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:33.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.567+0000 7fb606771640 10 monclient(hunting): get_auth_request con 0x7fb60013f490 auth_method 0 2024-03-13T07:34:33.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.567+0000 7fb606771640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:33.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.568+0000 7fb606771640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:33.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.568+0000 7fb606771640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:33.570 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.568+0000 7fb60576f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:33.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.568+0000 7fb60576f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:33.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.568+0000 7fb60576f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:33.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.568+0000 7fb606771640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:33.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.568+0000 7fb606771640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:33.571 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.568+0000 7fb606771640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:33.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.570+0000 7fb60576f640 10 monclient(hunting): handle_auth_done global_id 4227 payload 274 2024-03-13T07:34:33.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.570+0000 7fb60576f640 1 -- 172.21.15.132:0/348854733 >> v2:172.21.15.195:3300/0 conn(0x7fb60013f490 msgr2=0x7fb60013f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:33.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.570+0000 7fb60576f640 1 --2- 172.21.15.132:0/348854733 >> v2:172.21.15.195:3300/0 conn(0x7fb60013f490 0x7fb60013f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:33.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.570+0000 7fb60576f640 1 -- 172.21.15.132:0/348854733 >> v1:172.21.15.195:6789/0 conn(0x7fb6000b5180 legacy=0x7fb600145ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:33.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.570+0000 7fb60576f640 10 monclient: _finish_hunting 0 2024-03-13T07:34:33.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.570+0000 7fb60576f640 1 monclient: found mon.noname-a 2024-03-13T07:34:33.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.570+0000 7fb60576f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:33.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.570+0000 7fb60576f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:33.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.570+0000 7fb60576f640 1 -- 172.21.15.132:0/348854733 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6000b47d0 con 0x7fb6000a96d0 2024-03-13T07:34:33.572 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.572+0000 7fb60576f640 1 --2- 172.21.15.132:0/348854733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6000a96d0 0x7fb6000aa210 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb600140130 tx=0x7fb5f002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62460587bd139cb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:33.574 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.574+0000 7fb604f6e640 1 -- 172.21.15.132:0/348854733 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00315a0 con 0x7fb6000a96d0 2024-03-13T07:34:33.574 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.574+0000 7fb604f6e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:33.574 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.574+0000 7fb604f6e640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:33.574 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.574+0000 7fb604f6e640 10 monclient: dump: 2024-03-13T07:34:33.574 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:33.574 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:33.574 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:33.575 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:33.575 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:33.575 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:33.575 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:33.575 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:33.575 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:33.575 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:33.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.574+0000 7fb604f6e640 10 monclient: _finish_auth 0 2024-03-13T07:34:33.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.574+0000 7fb604f6e640 10 monclient: _check_auth_tickets 2024-03-13T07:34:33.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.574+0000 7fb604f6e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:33.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.574+0000 7fb6081d4640 5 monclient: authenticate success, global_id 4227 2024-03-13T07:34:33.575 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.574+0000 7fb6081d4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:33.576 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.576+0000 7fb604f6e640 1 -- 172.21.15.132:0/348854733 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f0037070 con 0x7fb6000a96d0 2024-03-13T07:34:33.576 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.576+0000 7fb604f6e640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:33.576 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.576+0000 7fb6081d4640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:33.576 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.576+0000 7fb6081d4640 4 set_mon_vals no callback set 2024-03-13T07:34:33.576 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.576+0000 7fb6081d4640 10 monclient: shutdown 2024-03-13T07:34:33.577 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.576+0000 7fb6081d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:33.577 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.576+0000 7fb6081d4640 1 -- 172.21.15.132:0/348854733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6000a96d0 msgr2=0x7fb6000aa210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:33.577 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.576+0000 7fb6081d4640 1 --2- 172.21.15.132:0/348854733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6000a96d0 0x7fb6000aa210 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb600140130 tx=0x7fb5f002b840 comp rx=0 tx=0).stop 2024-03-13T07:34:33.577 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.576+0000 7fb6081d4640 1 -- 172.21.15.132:0/348854733 shutdown_connections 2024-03-13T07:34:33.577 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.576+0000 7fb6081d4640 1 --2- 172.21.15.132:0/348854733 >> v2:172.21.15.195:3300/0 conn(0x7fb60013f490 0x7fb60013f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:33.577 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.576+0000 7fb6081d4640 1 --2- 172.21.15.132:0/348854733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6000a96d0 0x7fb6000aa210 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:33.577 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.576+0000 7fb6081d4640 1 -- 172.21.15.132:0/348854733 >> 172.21.15.132:0/348854733 conn(0x7fb600072750 msgr2=0x7fb600072b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:33.577 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.577+0000 7fb6081d4640 1 -- 172.21.15.132:0/348854733 shutdown_connections 2024-03-13T07:34:33.577 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.577+0000 7fb6081d4640 1 -- 172.21.15.132:0/348854733 wait complete. 2024-03-13T07:34:33.582 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.578+0000 7fb6081d4640 10 monclient: build_initial_monmap 2024-03-13T07:34:33.582 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.578+0000 7fb6081d4640 10 monclient: monmap: 2024-03-13T07:34:33.582 INFO:teuthology.misc.health.smithi132.stderr:epoch 0 2024-03-13T07:34:33.582 INFO:teuthology.misc.health.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:33.582 INFO:teuthology.misc.health.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:33.582 INFO:teuthology.misc.health.smithi132.stderr:created 0.000000 2024-03-13T07:34:33.582 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:33.582 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:33.582 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:33.582 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:33.582 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:33.582 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:33.583 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.578+0000 7fb6081d4640 1 librados: starting msgr at 2024-03-13T07:34:33.583 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.578+0000 7fb6081d4640 1 librados: starting objecter 2024-03-13T07:34:33.583 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.578+0000 7fb6081d4640 1 Processor -- start 2024-03-13T07:34:33.583 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.578+0000 7fb6081d4640 1 -- start start 2024-03-13T07:34:33.583 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.578+0000 7fb6081d4640 1 librados: setting wanted keys 2024-03-13T07:34:33.583 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.578+0000 7fb6081d4640 1 librados: calling monclient init 2024-03-13T07:34:33.583 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.578+0000 7fb6081d4640 10 monclient: init 2024-03-13T07:34:33.583 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb6081d4640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:33.583 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb6081d4640 10 monclient: _add_conns ranks=[2,1,0] 2024-03-13T07:34:33.583 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb6081d4640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6000a96d0 0x7fb600070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:33.583 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb6081d4640 10 monclient(hunting): picked mon.noname-c con 0x7fb6000a96d0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:33.583 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb6081d4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6000b5180 0x7fb60006e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:33.583 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb6081d4640 10 monclient(hunting): picked mon.noname-b con 0x7fb6000b5180 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:33.583 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb6081d4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb60013f490 0x7fb600070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:33.584 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb6081d4640 10 monclient(hunting): picked mon.noname-a con 0x7fb60013f490 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:33.584 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb6081d4640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:33.584 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb6081d4640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60013fe20 con 0x7fb60013f490 2024-03-13T07:34:33.584 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb6081d4640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:33.584 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb6081d4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb600146230 con 0x7fb6000b5180 2024-03-13T07:34:33.584 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb6081d4640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:33.584 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb6081d4640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60014ce70 con 0x7fb6000a96d0 2024-03-13T07:34:33.584 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb6081d4640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:33.584 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb6081d4640 10 monclient(hunting): authenticate will time out at 1300.987397s 2024-03-13T07:34:33.584 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.579+0000 7fb605f70640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6000a96d0 0x7fb600070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:33.584 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.580+0000 7fb606771640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb60013f490 0x7fb600070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:33.584 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.580+0000 7fb60576f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6000b5180 0x7fb60006e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:33.584 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.580+0000 7fb606771640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb60013f490 0x7fb600070f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35598/0 (socket says 172.21.15.132:35598) 2024-03-13T07:34:33.584 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.580+0000 7fb606771640 1 -- 172.21.15.132:0/894462691 learned_addr learned my addr 172.21.15.132:0/894462691 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:33.584 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.580+0000 7fb606771640 10 monclient(hunting): get_auth_request con 0x7fb60013f490 auth_method 0 2024-03-13T07:34:33.585 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.580+0000 7fb606771640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:33.585 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.580+0000 7fb606771640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:33.585 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.580+0000 7fb606771640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:33.585 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.580+0000 7fb60576f640 10 monclient(hunting): get_auth_request con 0x7fb6000b5180 auth_method 0 2024-03-13T07:34:33.585 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.580+0000 7fb60576f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:33.585 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.580+0000 7fb60576f640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:33.585 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.580+0000 7fb60576f640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:33.585 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.580+0000 7fb605f70640 10 monclient(hunting): get_auth_request con 0x7fb6000a96d0 auth_method 0 2024-03-13T07:34:33.585 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.580+0000 7fb605f70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:33.585 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.580+0000 7fb605f70640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:33.585 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.580+0000 7fb605f70640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:33.585 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.581+0000 7fb606771640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:33.585 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.581+0000 7fb606771640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:33.585 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.581+0000 7fb606771640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:33.585 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.581+0000 7fb60576f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:33.586 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.581+0000 7fb60576f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:33.586 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.581+0000 7fb60576f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:33.586 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.581+0000 7fb605f70640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:33.586 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.581+0000 7fb605f70640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:33.586 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.581+0000 7fb605f70640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:33.587 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.583+0000 7fb60576f640 10 monclient(hunting): handle_auth_done global_id 4237 payload 824 2024-03-13T07:34:33.587 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.583+0000 7fb60576f640 1 -- 172.21.15.132:0/894462691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6000a96d0 msgr2=0x7fb600070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:33.587 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.583+0000 7fb60576f640 1 --2- 172.21.15.132:0/894462691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6000a96d0 0x7fb600070790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:33.587 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.583+0000 7fb60576f640 1 -- 172.21.15.132:0/894462691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb60013f490 msgr2=0x7fb600070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:33.587 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.583+0000 7fb60576f640 1 --2- 172.21.15.132:0/894462691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb60013f490 0x7fb600070f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:33.587 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.583+0000 7fb60576f640 10 monclient: _finish_hunting 0 2024-03-13T07:34:33.587 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.583+0000 7fb60576f640 1 monclient: found mon.noname-b 2024-03-13T07:34:33.587 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.583+0000 7fb60576f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:33.587 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.583+0000 7fb60576f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:33.587 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.583+0000 7fb60576f640 1 -- 172.21.15.132:0/894462691 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb600070090 con 0x7fb6000b5180 2024-03-13T07:34:33.587 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.585+0000 7fb60576f640 1 --2- 172.21.15.132:0/894462691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6000b5180 0x7fb60006e1c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0033000 tx=0x7fb5f00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:33.588 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.586+0000 7fb5f67fc640 10 client.?.objecter ms_handle_connect 0x7fb6000b5180 2024-03-13T07:34:33.588 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.586+0000 7fb5f67fc640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:33.591 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb5f67fc640 1 -- 172.21.15.132:0/894462691 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0006e10 con 0x7fb6000b5180 2024-03-13T07:34:33.591 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb5f67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:33.591 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb5f67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:33.591 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb5f67fc640 10 monclient: dump: 2024-03-13T07:34:33.591 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:33.591 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:33.591 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:33.591 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:33.591 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:33.591 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:33.591 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:33.592 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:33.592 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:33.592 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:33.592 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb5f67fc640 10 monclient: _finish_auth 0 2024-03-13T07:34:33.592 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb5f67fc640 10 monclient: _check_auth_tickets 2024-03-13T07:34:33.592 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb5f67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:33.592 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb5f67fc640 1 -- 172.21.15.132:0/894462691 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f00317a0 con 0x7fb6000b5180 2024-03-13T07:34:33.592 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb6081d4640 5 monclient: authenticate success, global_id 4237 2024-03-13T07:34:33.592 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb6081d4640 10 monclient: _renew_subs 2024-03-13T07:34:33.592 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb6081d4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:33.592 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb6081d4640 1 -- 172.21.15.132:0/894462691 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb600070370 con 0x7fb6000b5180 2024-03-13T07:34:33.592 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb5f67fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:33.592 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb5f67fc640 1 -- 172.21.15.132:0/894462691 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0031b90 con 0x7fb6000b5180 2024-03-13T07:34:33.592 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb5f67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:33.592 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb5f67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:33.593 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb5f67fc640 10 monclient: dump: 2024-03-13T07:34:33.593 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:33.593 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:33.593 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:33.593 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:33.593 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:33.593 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:33.593 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:33.593 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:33.593 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:33.593 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:33.593 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.589+0000 7fb5f7fff640 4 set_mon_vals no callback set 2024-03-13T07:34:33.593 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.590+0000 7fb6081d4640 10 client.4237.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:33.593 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.590+0000 7fb6081d4640 10 monclient: _renew_subs 2024-03-13T07:34:33.593 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.590+0000 7fb6081d4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:33.593 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.590+0000 7fb6081d4640 1 -- 172.21.15.132:0/894462691 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60006e7c0 con 0x7fb6000b5180 2024-03-13T07:34:33.594 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.590+0000 7fb6081d4640 1 librados: init done 2024-03-13T07:34:33.594 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.591+0000 7fb6081d4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:33.594 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.591+0000 7fb6081d4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:33.594 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.591+0000 7fb6081d4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:33.594 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.591+0000 7fb6081d4640 1 -- 172.21.15.132:0/894462691 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60006a510 con 0x7fb6000b5180 2024-03-13T07:34:33.597 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-03-13T07:34:33.593+0000 bc8e640 -1 osd.3 0 waiting for initial osdmap 2024-03-13T07:34:33.611 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.610+0000 7fb5f67fc640 1 -- 172.21.15.132:0/894462691 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 4) v1 ==== 90080+0+0 (secure 0 0 0) 0x7fb5f0031d30 con 0x7fb6000b5180 2024-03-13T07:34:33.611 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.611+0000 7fb5f67fc640 1 --2- 172.21.15.132:0/894462691 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb5e406d520 0x7fb5e406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:33.611 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.611+0000 7fb605f70640 1 --2- 172.21.15.132:0/894462691 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb5e406d520 0x7fb5e406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:33.611 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.611+0000 7fb605f70640 10 monclient: get_auth_request con 0x7fb5e406d520 auth_method 0 2024-03-13T07:34:33.612 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.612+0000 7fb5f67fc640 10 client.4237.objecter ms_handle_connect 0x7fb5e406d520 2024-03-13T07:34:33.612 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.612+0000 7fb605f70640 1 --2- 172.21.15.132:0/894462691 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb5e406d520 0x7fb5e406f9e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc005880 tx=0x7fb5fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:33.613 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.613+0000 7fb5f67fc640 1 -- 172.21.15.132:0/894462691 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2791+0+0 (secure 0 0 0) 0x7fb5f002f070 con 0x7fb6000b5180 2024-03-13T07:34:33.613 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.613+0000 7fb5f67fc640 10 client.4237.objecter ms_dispatch 0x7fb600000b90 osd_map(14..14 src has 1..14) v4 2024-03-13T07:34:33.613 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.613+0000 7fb5f67fc640 3 client.4237.objecter handle_osd_map got epochs [14,14] > 0 2024-03-13T07:34:33.613 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.613+0000 7fb5f67fc640 3 client.4237.objecter handle_osd_map decoding full epoch 14 2024-03-13T07:34:33.614 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.613+0000 7fb5f67fc640 20 client.4237.objecter dump_active .. 0 homeless 2024-03-13T07:34:33.859 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.859+0000 7fb5f67fc640 1 -- 172.21.15.132:0/894462691 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fb5f007b8c0 con 0x7fb6000b5180 2024-03-13T07:34:33.859 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.859+0000 7fb5f67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:33.859 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:33.859+0000 7fb5f67fc640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:34.090 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-03-13T07:34:34.071+0000 10255640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T07:34:34.129 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.129+0000 7fb6081d4640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:34.129 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.129+0000 7fb6081d4640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:34.129 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.129+0000 7fb6081d4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:34.129 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.129+0000 7fb6081d4640 1 -- 172.21.15.132:0/894462691 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb600067ca0 con 0x7fb6000b5180 2024-03-13T07:34:34.155 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.155+0000 7fb5f67fc640 1 -- 172.21.15.132:0/894462691 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+8076 (secure 0 0 0) 0x7fb5f007f960 con 0x7fb6000b5180 2024-03-13T07:34:34.155 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.155+0000 7fb5f67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:34.155 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.155+0000 7fb5f67fc640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:34.156 INFO:teuthology.misc.health.smithi132.stdout: 2024-03-13T07:34:34.156 INFO:teuthology.misc.health.smithi132.stdout:{"epoch":14,"fsid":"74e4bc2f-6a5e-4423-a8c8-807970a2f96c","created":"2024-03-13T07:33:47.494293+0000","modified":"2024-03-13T07:34:32.976241+0000","last_up_change":"2024-03-13T07:34:31.795731+0000","last_in_change":"2024-03-13T07:33:53.171597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0edb216c-bc1e-4865-aafe-218611f3eb09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6be0ae6b-951e-4279-ba0a-9b05b6e6e85e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"18008b7f-cfe7-4450-a3cf-aab63ebbe99d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"411b6066-4b9f-40e7-b92d-4ba914eab63e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1a719246-ae04-42b0-bd9c-fc318dc57ecb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d9ee991-241b-4507-ba2f-e3b5e2844d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6817","nonce":1953907706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6819","nonce":1953907706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6825","nonce":1953907706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6821","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6822","nonce":1953907706}]},"public_addr":"172.21.15.195:6817/1953907706","cluster_addr":"172.21.15.195:6819/1953907706","heartbeat_back_addr":"172.21.15.195:6825/1953907706","heartbeat_front_addr":"172.21.15.195:6822/1953907706","state":["exists","up"]},{"osd":6,"uuid":"af8b34fd-927b-4860-971b-9fddf699a0dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"19a39645-8664-463b-bae3-d7aa1026e5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6823","nonce":3378300314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6827","nonce":3378300314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6831","nonce":3378300314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6829","nonce":3378300314}]},"public_addr":"172.21.15.195:6823/3378300314","cluster_addr":"172.21.15.195:6827/3378300314","heartbeat_back_addr":"172.21.15.195:6831/3378300314","heartbeat_front_addr":"172.21.15.195:6829/3378300314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:30.243386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T07:34:34.159 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.158+0000 7fb6081d4640 10 librados: watch_flush enter 2024-03-13T07:34:34.159 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.158+0000 7fb6081d4640 10 librados: watch_flush exit 2024-03-13T07:34:34.159 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 20 client.4237.objecter shutdown clearing up homeless session... 2024-03-13T07:34:34.159 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 10 client.4237.objecter successfully canceled tick 2024-03-13T07:34:34.159 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 1 -- 172.21.15.132:0/894462691 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb5e406d520 msgr2=0x7fb5e406f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:34.159 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 1 --2- 172.21.15.132:0/894462691 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb5e406d520 0x7fb5e406f9e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc005880 tx=0x7fb5fc009040 comp rx=0 tx=0).stop 2024-03-13T07:34:34.159 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 10 monclient: shutdown 2024-03-13T07:34:34.160 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:34.160 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 1 -- 172.21.15.132:0/894462691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6000b5180 msgr2=0x7fb60006e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:34.160 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 1 --2- 172.21.15.132:0/894462691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6000b5180 0x7fb60006e1c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0033000 tx=0x7fb5f00046c0 comp rx=0 tx=0).stop 2024-03-13T07:34:34.160 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 1 -- 172.21.15.132:0/894462691 shutdown_connections 2024-03-13T07:34:34.160 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 1 --2- 172.21.15.132:0/894462691 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb5e406d520 0x7fb5e406f9e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:34.160 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 1 --2- 172.21.15.132:0/894462691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb60013f490 0x7fb600070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:34.160 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 1 --2- 172.21.15.132:0/894462691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6000b5180 0x7fb60006e1c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:34.160 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 1 --2- 172.21.15.132:0/894462691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6000a96d0 0x7fb600070790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:34.160 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 1 -- 172.21.15.132:0/894462691 >> 172.21.15.132:0/894462691 conn(0x7fb600072750 msgr2=0x7fb6000b7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:34.160 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 1 -- 172.21.15.132:0/894462691 shutdown_connections 2024-03-13T07:34:34.160 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 1 -- 172.21.15.132:0/894462691 wait complete. 2024-03-13T07:34:34.160 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:34.159+0000 7fb6081d4640 1 librados: shutdown 2024-03-13T07:34:34.174 DEBUG:teuthology.misc:2 of 8 OSDs are up 2024-03-13T07:34:38.119 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T07:34:37.983+0000 5dc3480 -1 bdev(0x17e76d30 /var/lib/ceph/osd/ceph-4/block) aio_submit retries 1 2024-03-13T07:34:38.119 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T07:34:38.073+0000 5dc3480 -1 bdev(0x17e76d30 /var/lib/ceph/osd/ceph-4/block) aio_submit retries 1 2024-03-13T07:34:38.374 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T07:34:38.119+0000 5dc3480 -1 bdev(0x17e76d30 /var/lib/ceph/osd/ceph-4/block) aio_submit retries 1 2024-03-13T07:34:38.413 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T07:34:38.379+0000 5dc3480 -1 bdev(0x17e76d30 /var/lib/ceph/osd/ceph-4/block) aio_submit retries 1 2024-03-13T07:34:39.845 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T07:34:39.836+0000 b47a640 -1 osd.4 0 waiting for initial osdmap 2024-03-13T07:34:40.287 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T07:34:40.265+0000 1425d640 -1 osd.4 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T07:34:40.805 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:40.642+0000 5dc3480 -1 bdev(0x1865c9b0 /var/lib/ceph/osd/ceph-1/block) aio_submit retries 1 2024-03-13T07:34:40.806 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:40.657+0000 5dc3480 -1 bdev(0x1865c9b0 /var/lib/ceph/osd/ceph-1/block) aio_submit retries 1 2024-03-13T07:34:40.806 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:40.713+0000 5dc3480 -1 bdev(0x1865c9b0 /var/lib/ceph/osd/ceph-1/block) aio_submit retries 1 2024-03-13T07:34:40.832 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:40.724+0000 5dc3480 -1 bdev(0x1865c9b0 /var/lib/ceph/osd/ceph-1/block) aio_submit retries 1 2024-03-13T07:34:40.958 INFO:tasks.ceph.osd.2.smithi132.stderr:2024-03-13T07:34:40.936+0000 5dc3480 -1 bdev(0x17e75d10 /var/lib/ceph/osd/ceph-2/block) aio_submit retries 2 2024-03-13T07:34:40.980 DEBUG:teuthology.orchestra.run.smithi132:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-13T07:34:41.053 INFO:tasks.ceph.osd.0.smithi132.stderr:2024-03-13T07:34:40.979+0000 5dc3480 -1 bdev(0x17e75d10 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 1 2024-03-13T07:34:41.083 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.082+0000 7f0b288c0640 10 monclient: get_monmap_and_config 2024-03-13T07:34:41.083 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.082+0000 7f0b288c0640 10 monclient: build_initial_monmap 2024-03-13T07:34:41.083 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.082+0000 7f0b288c0640 10 monclient: monmap: 2024-03-13T07:34:41.083 INFO:teuthology.misc.health.smithi132.stderr:epoch 0 2024-03-13T07:34:41.083 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:41.084 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:34:41.083693+0000 2024-03-13T07:34:41.084 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:34:41.083693+0000 2024-03-13T07:34:41.084 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:41.084 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:41.084 INFO:teuthology.misc.health.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:41.084 INFO:teuthology.misc.health.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:41.084 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:41.084 INFO:teuthology.misc.health.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:41.084 INFO:teuthology.misc.health.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:41.084 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:41.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.083+0000 7f0b288c0640 1 Processor -- start 2024-03-13T07:34:41.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.083+0000 7f0b288c0640 1 -- start start 2024-03-13T07:34:41.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.083+0000 7f0b288c0640 10 monclient: init 2024-03-13T07:34:41.086 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.084+0000 7f0b288c0640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:41.086 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.084+0000 7f0b288c0640 10 monclient: _add_conns ranks=[4,3,1,2,0] 2024-03-13T07:34:41.086 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.084+0000 7f0b288c0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0b2406fb00 0x7f0b2406fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:41.086 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.084+0000 7f0b288c0640 10 monclient(hunting): picked mon.noname-c con 0x7f0b2406fb00 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:41.086 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.084+0000 7f0b288c0640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0b2406f160 0x7f0b240ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:41.086 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.084+0000 7f0b288c0640 10 monclient(hunting): picked mon.noname-a con 0x7f0b2406f160 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:41.086 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.084+0000 7f0b288c0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0b2413f1e0 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:41.086 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.084+0000 7f0b288c0640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b24071170 con 0x7f0b2413f1e0 2024-03-13T07:34:41.086 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.084+0000 7f0b288c0640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:41.087 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.084+0000 7f0b288c0640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b24071870 con 0x7f0b2406f160 2024-03-13T07:34:41.087 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.084+0000 7f0b288c0640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:41.087 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.084+0000 7f0b288c0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b24071450 con 0x7f0b2406fb00 2024-03-13T07:34:41.087 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.084+0000 7f0b288c0640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:41.087 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.084+0000 7f0b288c0640 10 monclient(hunting): authenticate will time out at 1308.492942s 2024-03-13T07:34:41.087 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.085+0000 7f0b2259c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0b2406fb00 0x7f0b2406fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:41.087 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.085+0000 7f0b2259c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0b2406fb00 0x7f0b2406fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:44650/0 (socket says 172.21.15.132:44650) 2024-03-13T07:34:41.087 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.085+0000 7f0b2259c640 1 -- 172.21.15.132:0/3841325875 learned_addr learned my addr 172.21.15.132:0/3841325875 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:41.087 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.085+0000 7f0b2259c640 10 monclient(hunting): get_auth_request con 0x7f0b2406fb00 auth_method 0 2024-03-13T07:34:41.087 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.085+0000 7f0b2259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:41.087 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.085+0000 7f0b2259c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:41.087 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.085+0000 7f0b2259c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:41.087 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.086+0000 7f0b2259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:41.087 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.086+0000 7f0b2259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:41.087 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.086+0000 7f0b2259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:41.088 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.088+0000 7f0b2259c640 10 monclient(hunting): handle_auth_done global_id 4243 payload 274 2024-03-13T07:34:41.088 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.088+0000 7f0b2259c640 1 -- 172.21.15.132:0/3841325875 >> v2:172.21.15.132:3300/0 conn(0x7f0b2406f160 msgr2=0x7f0b240ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:41.088 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.088+0000 7f0b2259c640 1 --2- 172.21.15.132:0/3841325875 >> v2:172.21.15.132:3300/0 conn(0x7f0b2406f160 0x7f0b240ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:41.088 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.088+0000 7f0b2259c640 1 -- 172.21.15.132:0/3841325875 >> v1:172.21.15.195:6789/0 conn(0x7f0b2413f1e0 legacy=0x7f0b24071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:41.088 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.088+0000 7f0b2259c640 10 monclient: _finish_hunting 0 2024-03-13T07:34:41.089 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.088+0000 7f0b2259c640 1 monclient: found mon.noname-c 2024-03-13T07:34:41.089 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.088+0000 7f0b2259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:41.089 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.088+0000 7f0b2259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T07:34:41.089 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.088+0000 7f0b2259c640 1 -- 172.21.15.132:0/3841325875 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b240aeb50 con 0x7f0b2406fb00 2024-03-13T07:34:41.091 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.091+0000 7f0b2259c640 1 --2- 172.21.15.132:0/3841325875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2406fb00 0x7f0b2406fee0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0b2406ce20 tx=0x7f0b1402e050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81331af07f912b17 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:41.096 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.095+0000 7f0b21d9b640 1 -- 172.21.15.132:0/3841325875 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1403b070 con 0x7f0b2406fb00 2024-03-13T07:34:41.096 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.095+0000 7f0b21d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:41.096 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.095+0000 7f0b21d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:41.096 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.095+0000 7f0b21d9b640 10 monclient: dump: 2024-03-13T07:34:41.096 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:41.096 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:41.096 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:41.097 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:41.097 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:41.097 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:41.097 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:41.097 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:41.097 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:41.097 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:41.097 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.095+0000 7f0b21d9b640 10 monclient: _finish_auth 0 2024-03-13T07:34:41.097 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.095+0000 7f0b21d9b640 10 monclient: _check_auth_tickets 2024-03-13T07:34:41.097 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.095+0000 7f0b21d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:41.097 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.095+0000 7f0b21d9b640 1 -- 172.21.15.132:0/3841325875 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1402ee00 con 0x7f0b2406fb00 2024-03-13T07:34:41.097 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.095+0000 7f0b21d9b640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:41.097 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.095+0000 7f0b21d9b640 1 -- 172.21.15.132:0/3841325875 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14036660 con 0x7f0b2406fb00 2024-03-13T07:34:41.097 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.095+0000 7f0b21d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:41.098 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.095+0000 7f0b21d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:41.098 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.095+0000 7f0b21d9b640 10 monclient: dump: 2024-03-13T07:34:41.098 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:41.098 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:41.098 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:41.098 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:41.098 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:41.098 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:41.098 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:41.098 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:41.098 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:41.098 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:41.098 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.095+0000 7f0b288c0640 5 monclient: authenticate success, global_id 4243 2024-03-13T07:34:41.098 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.095+0000 7f0b288c0640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:41.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.095+0000 7f0b288c0640 4 set_mon_vals no callback set 2024-03-13T07:34:41.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.096+0000 7f0b288c0640 10 monclient: shutdown 2024-03-13T07:34:41.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.096+0000 7f0b288c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:41.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.096+0000 7f0b288c0640 1 -- 172.21.15.132:0/3841325875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2406fb00 msgr2=0x7f0b2406fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:41.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.096+0000 7f0b288c0640 1 --2- 172.21.15.132:0/3841325875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2406fb00 0x7f0b2406fee0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0b2406ce20 tx=0x7f0b1402e050 comp rx=0 tx=0).stop 2024-03-13T07:34:41.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.096+0000 7f0b288c0640 1 -- 172.21.15.132:0/3841325875 shutdown_connections 2024-03-13T07:34:41.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.096+0000 7f0b288c0640 1 --2- 172.21.15.132:0/3841325875 >> v2:172.21.15.132:3300/0 conn(0x7f0b2406f160 0x7f0b240ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:41.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.096+0000 7f0b288c0640 1 --2- 172.21.15.132:0/3841325875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2406fb00 0x7f0b2406fee0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:41.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.096+0000 7f0b288c0640 1 -- 172.21.15.132:0/3841325875 >> 172.21.15.132:0/3841325875 conn(0x7f0b240b3df0 msgr2=0x7f0b240b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:41.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.096+0000 7f0b288c0640 1 -- 172.21.15.132:0/3841325875 shutdown_connections 2024-03-13T07:34:41.099 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.096+0000 7f0b288c0640 1 -- 172.21.15.132:0/3841325875 wait complete. 2024-03-13T07:34:41.103 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.097+0000 7f0b288c0640 10 monclient: build_initial_monmap 2024-03-13T07:34:41.103 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.097+0000 7f0b288c0640 10 monclient: monmap: 2024-03-13T07:34:41.103 INFO:teuthology.misc.health.smithi132.stderr:epoch 0 2024-03-13T07:34:41.103 INFO:teuthology.misc.health.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:41.104 INFO:teuthology.misc.health.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:41.104 INFO:teuthology.misc.health.smithi132.stderr:created 0.000000 2024-03-13T07:34:41.104 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:41.104 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:41.104 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:41.104 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:41.104 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:41.104 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:41.104 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.097+0000 7f0b288c0640 1 librados: starting msgr at 2024-03-13T07:34:41.104 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.097+0000 7f0b288c0640 1 librados: starting objecter 2024-03-13T07:34:41.104 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.097+0000 7f0b288c0640 1 Processor -- start 2024-03-13T07:34:41.104 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.098+0000 7f0b288c0640 1 -- start start 2024-03-13T07:34:41.104 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.098+0000 7f0b288c0640 1 librados: setting wanted keys 2024-03-13T07:34:41.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.098+0000 7f0b288c0640 1 librados: calling monclient init 2024-03-13T07:34:41.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.098+0000 7f0b288c0640 10 monclient: init 2024-03-13T07:34:41.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.098+0000 7f0b288c0640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:41.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.098+0000 7f0b288c0640 10 monclient: _add_conns ranks=[2,0,1] 2024-03-13T07:34:41.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.098+0000 7f0b288c0640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b2406f160 0x7f0b24137e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:41.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.098+0000 7f0b288c0640 10 monclient(hunting): picked mon.noname-c con 0x7f0b2406f160 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:41.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.098+0000 7f0b288c0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b2406fb00 0x7f0b241383b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:41.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.098+0000 7f0b288c0640 10 monclient(hunting): picked mon.noname-a con 0x7f0b2406fb00 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:41.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.099+0000 7f0b288c0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2413f1e0 0x7f0b24140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:41.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.099+0000 7f0b288c0640 10 monclient(hunting): picked mon.noname-b con 0x7f0b2413f1e0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:41.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.099+0000 7f0b288c0640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:41.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.099+0000 7f0b288c0640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b24072570 con 0x7f0b2406fb00 2024-03-13T07:34:41.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.099+0000 7f0b288c0640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:41.105 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.099+0000 7f0b288c0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240723f0 con 0x7f0b2413f1e0 2024-03-13T07:34:41.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.099+0000 7f0b288c0640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:41.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.099+0000 7f0b288c0640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2413f630 con 0x7f0b2406f160 2024-03-13T07:34:41.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.099+0000 7f0b288c0640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:41.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.099+0000 7f0b288c0640 10 monclient(hunting): authenticate will time out at 1308.507582s 2024-03-13T07:34:41.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.100+0000 7f0b2259c640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b2406f160 0x7f0b24137e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:41.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.100+0000 7f0b22d9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2413f1e0 0x7f0b24140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:41.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.100+0000 7f0b2259c640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b2406f160 0x7f0b24137e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:49704/0 (socket says 172.21.15.132:49704) 2024-03-13T07:34:41.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.100+0000 7f0b2259c640 1 -- 172.21.15.132:0/1209450039 learned_addr learned my addr 172.21.15.132:0/1209450039 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:41.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.100+0000 7f0b2259c640 10 monclient(hunting): get_auth_request con 0x7f0b2406f160 auth_method 0 2024-03-13T07:34:41.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.100+0000 7f0b2259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:41.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.100+0000 7f0b2259c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:41.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.100+0000 7f0b2259c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:41.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.100+0000 7f0b1bfff640 1 --2- 172.21.15.132:0/1209450039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b2406fb00 0x7f0b241383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:41.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.100+0000 7f0b22d9d640 10 monclient(hunting): get_auth_request con 0x7f0b2413f1e0 auth_method 0 2024-03-13T07:34:41.106 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.100+0000 7f0b22d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:41.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.100+0000 7f0b22d9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:41.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.100+0000 7f0b22d9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:41.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.101+0000 7f0b1bfff640 10 monclient(hunting): get_auth_request con 0x7f0b2406fb00 auth_method 0 2024-03-13T07:34:41.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.101+0000 7f0b1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:41.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.101+0000 7f0b1bfff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:41.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.101+0000 7f0b1bfff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:41.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.101+0000 7f0b22d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:41.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.101+0000 7f0b22d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:41.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.101+0000 7f0b22d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:41.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.101+0000 7f0b2259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:41.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.101+0000 7f0b2259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:41.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.101+0000 7f0b2259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:41.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.101+0000 7f0b1bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:41.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.101+0000 7f0b1bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:41.107 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.101+0000 7f0b1bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:41.108 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:40.833+0000 5dc3480 -1 bdev(0x1865c9b0 /var/lib/ceph/osd/ceph-1/block) aio_submit retries 1 2024-03-13T07:34:41.108 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:40.942+0000 5dc3480 -1 bdev(0x1865c9b0 /var/lib/ceph/osd/ceph-1/block) aio_submit retries 1 2024-03-13T07:34:41.109 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.104+0000 7f0b22d9d640 10 monclient(hunting): handle_auth_done global_id 4246 payload 824 2024-03-13T07:34:41.109 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.104+0000 7f0b22d9d640 1 -- 172.21.15.132:0/1209450039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b2406f160 msgr2=0x7f0b24137e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:41.109 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.104+0000 7f0b22d9d640 1 --2- 172.21.15.132:0/1209450039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b2406f160 0x7f0b24137e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:41.109 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.104+0000 7f0b22d9d640 1 -- 172.21.15.132:0/1209450039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b2406fb00 msgr2=0x7f0b241383b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:41.110 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.104+0000 7f0b22d9d640 1 --2- 172.21.15.132:0/1209450039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b2406fb00 0x7f0b241383b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:41.110 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.104+0000 7f0b22d9d640 10 monclient: _finish_hunting 0 2024-03-13T07:34:41.110 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.104+0000 7f0b22d9d640 1 monclient: found mon.noname-b 2024-03-13T07:34:41.110 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.104+0000 7f0b22d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:41.110 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.104+0000 7f0b22d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:41.110 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.104+0000 7f0b22d9d640 1 -- 172.21.15.132:0/1209450039 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b241425b0 con 0x7f0b2413f1e0 2024-03-13T07:34:41.110 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.107+0000 7f0b22d9d640 1 --2- 172.21.15.132:0/1209450039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2413f1e0 0x7f0b24140910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00c760 tx=0x7f0b1c00cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:41.110 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.107+0000 7f0b1affd640 10 client.?.objecter ms_handle_connect 0x7f0b2413f1e0 2024-03-13T07:34:41.110 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.107+0000 7f0b1affd640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:41.111 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.111+0000 7f0b1affd640 1 -- 172.21.15.132:0/1209450039 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c016020 con 0x7f0b2413f1e0 2024-03-13T07:34:41.111 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.111+0000 7f0b1affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:41.111 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.111+0000 7f0b1affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:41.111 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.111+0000 7f0b1affd640 10 monclient: dump: 2024-03-13T07:34:41.111 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:41.111 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:41.111 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:41.111 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:41.111 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:41.111 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:41.111 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:41.111 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:41.112 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:41.112 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:41.112 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.111+0000 7f0b1affd640 10 monclient: _finish_auth 0 2024-03-13T07:34:41.112 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.111+0000 7f0b1affd640 10 monclient: _check_auth_tickets 2024-03-13T07:34:41.112 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.111+0000 7f0b1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:41.112 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.111+0000 7f0b1affd640 1 -- 172.21.15.132:0/1209450039 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1c006d50 con 0x7f0b2413f1e0 2024-03-13T07:34:41.112 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.111+0000 7f0b1affd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:41.112 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.111+0000 7f0b2159a640 4 set_mon_vals no callback set 2024-03-13T07:34:41.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.112+0000 7f0b288c0640 5 monclient: authenticate success, global_id 4246 2024-03-13T07:34:41.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.112+0000 7f0b1affd640 1 -- 172.21.15.132:0/1209450039 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c003bb0 con 0x7f0b2413f1e0 2024-03-13T07:34:41.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.112+0000 7f0b1affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:41.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.112+0000 7f0b1affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:41.113 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.112+0000 7f0b1affd640 10 monclient: dump: 2024-03-13T07:34:41.113 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:41.113 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:41.113 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:41.113 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:41.113 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:41.113 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:41.114 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:41.114 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:41.114 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:41.114 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:41.114 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.112+0000 7f0b288c0640 10 monclient: _renew_subs 2024-03-13T07:34:41.114 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.112+0000 7f0b288c0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:41.114 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.112+0000 7f0b288c0640 1 -- 172.21.15.132:0/1209450039 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b2413fee0 con 0x7f0b2413f1e0 2024-03-13T07:34:41.114 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.112+0000 7f0b288c0640 10 client.4246.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:41.114 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.112+0000 7f0b288c0640 10 monclient: _renew_subs 2024-03-13T07:34:41.114 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.112+0000 7f0b288c0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:41.114 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.112+0000 7f0b288c0640 1 -- 172.21.15.132:0/1209450039 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b24141030 con 0x7f0b2413f1e0 2024-03-13T07:34:41.114 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.112+0000 7f0b288c0640 1 librados: init done 2024-03-13T07:34:41.114 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.113+0000 7f0b288c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:41.114 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.113+0000 7f0b288c0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:41.115 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.113+0000 7f0b288c0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:41.115 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.113+0000 7f0b288c0640 1 -- 172.21.15.132:0/1209450039 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b2406a510 con 0x7f0b2413f1e0 2024-03-13T07:34:41.130 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.131+0000 7f0b1affd640 1 -- 172.21.15.132:0/1209450039 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f0b1c00a850 con 0x7f0b2413f1e0 2024-03-13T07:34:41.131 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.131+0000 7f0b1affd640 1 --2- 172.21.15.132:0/1209450039 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f0b0406d560 0x7f0b0406fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:41.131 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.131+0000 7f0b2259c640 1 --2- 172.21.15.132:0/1209450039 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f0b0406d560 0x7f0b0406fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:41.131 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.131+0000 7f0b2259c640 10 monclient: get_auth_request con 0x7f0b0406d560 auth_method 0 2024-03-13T07:34:41.132 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.132+0000 7f0b2259c640 1 --2- 172.21.15.132:0/1209450039 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f0b0406d560 0x7f0b0406fa20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400b8d0 tx=0x7f0b14032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:41.132 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.132+0000 7f0b1affd640 10 client.4246.objecter ms_handle_connect 0x7f0b0406d560 2024-03-13T07:34:41.134 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:41.064+0000 5dc3480 -1 bdev(0x1865c9b0 /var/lib/ceph/osd/ceph-1/block) aio_submit retries 1 2024-03-13T07:34:41.145 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:41.104+0000 5dc3480 -1 bdev(0x1865c9b0 /var/lib/ceph/osd/ceph-1/block) aio_submit retries 1 2024-03-13T07:34:41.146 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:41.136+0000 5dc3480 -1 bdev(0x1865c9b0 /var/lib/ceph/osd/ceph-1/block) aio_submit retries 1 2024-03-13T07:34:41.236 INFO:tasks.ceph.osd.2.smithi132.stderr:2024-03-13T07:34:41.119+0000 5dc3480 -1 bdev(0x17e75d10 /var/lib/ceph/osd/ceph-2/block) aio_submit retries 1 2024-03-13T07:34:41.236 INFO:tasks.ceph.osd.2.smithi132.stderr:2024-03-13T07:34:41.137+0000 5dc3480 -1 bdev(0x17e75d10 /var/lib/ceph/osd/ceph-2/block) aio_submit retries 1 2024-03-13T07:34:41.282 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:41.146+0000 5dc3480 -1 bdev(0x1865c9b0 /var/lib/ceph/osd/ceph-1/block) aio_submit retries 1 2024-03-13T07:34:41.282 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:41.219+0000 5dc3480 -1 bdev(0x1865c9b0 /var/lib/ceph/osd/ceph-1/block) aio_submit retries 1 2024-03-13T07:34:41.378 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.378+0000 7f0b1affd640 1 -- 172.21.15.132:0/1209450039 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3737+0+0 (secure 0 0 0) 0x7f0b1c056e20 con 0x7f0b2413f1e0 2024-03-13T07:34:41.378 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.378+0000 7f0b1affd640 10 client.4246.objecter ms_dispatch 0x7f0b24000b90 osd_map(17..17 src has 1..17) v4 2024-03-13T07:34:41.378 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.378+0000 7f0b1affd640 3 client.4246.objecter handle_osd_map got epochs [17,17] > 0 2024-03-13T07:34:41.378 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.378+0000 7f0b1affd640 3 client.4246.objecter handle_osd_map decoding full epoch 17 2024-03-13T07:34:41.378 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.378+0000 7f0b1affd640 20 client.4246.objecter dump_active .. 0 homeless 2024-03-13T07:34:41.387 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.388+0000 7f0b1affd640 1 -- 172.21.15.132:0/1209450039 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f0b1c058d80 con 0x7f0b2413f1e0 2024-03-13T07:34:41.387 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.388+0000 7f0b1affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:41.387 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.388+0000 7f0b1affd640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:41.406 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:41.380+0000 5dc3480 -1 bdev(0x1865c9b0 /var/lib/ceph/osd/ceph-1/block) aio_submit retries 1 2024-03-13T07:34:41.415 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:41.408+0000 5dc3480 -1 bdev(0x1865c9b0 /var/lib/ceph/osd/ceph-1/block) aio_submit retries 1 2024-03-13T07:34:41.417 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:41.416+0000 5dc3480 -1 bdev(0x1865c9b0 /var/lib/ceph/osd/ceph-1/block) aio_submit retries 1 2024-03-13T07:34:41.602 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.602+0000 7f0b288c0640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:41.602 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.602+0000 7f0b288c0640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:41.602 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.602+0000 7f0b288c0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:41.602 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.602+0000 7f0b288c0640 1 -- 172.21.15.132:0/1209450039 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b24140110 con 0x7f0b2413f1e0 2024-03-13T07:34:41.662 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:41.611+0000 5dc3480 -1 bdev(0x1865c9b0 /var/lib/ceph/osd/ceph-1/block) aio_submit retries 3 2024-03-13T07:34:41.688 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.689+0000 7f0b1affd640 1 -- 172.21.15.132:0/1209450039 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+10955 (secure 0 0 0) 0x7f0b24140110 con 0x7f0b2413f1e0 2024-03-13T07:34:41.689 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.689+0000 7f0b1affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:41.689 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.689+0000 7f0b1affd640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:41.689 INFO:teuthology.misc.health.smithi132.stdout: 2024-03-13T07:34:41.689 INFO:teuthology.misc.health.smithi132.stdout:{"epoch":17,"fsid":"74e4bc2f-6a5e-4423-a8c8-807970a2f96c","created":"2024-03-13T07:33:47.494293+0000","modified":"2024-03-13T07:34:36.293065+0000","last_up_change":"2024-03-13T07:34:35.246123+0000","last_in_change":"2024-03-13T07:33:53.171597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-13T07:34:33.504752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.690000057220459,"score_stable":7.690000057220459,"optimal_score":0.12999999523162842,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0edb216c-bc1e-4865-aafe-218611f3eb09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6be0ae6b-951e-4279-ba0a-9b05b6e6e85e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"18008b7f-cfe7-4450-a3cf-aab63ebbe99d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"411b6066-4b9f-40e7-b92d-4ba914eab63e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6811","nonce":1770057764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6813","nonce":1770057764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6817","nonce":1770057764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6815","nonce":1770057764}]},"public_addr":"172.21.15.132:6811/1770057764","cluster_addr":"172.21.15.132:6813/1770057764","heartbeat_back_addr":"172.21.15.132:6817/1770057764","heartbeat_front_addr":"172.21.15.132:6815/1770057764","state":["exists","up"]},{"osd":4,"uuid":"1a719246-ae04-42b0-bd9c-fc318dc57ecb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d9ee991-241b-4507-ba2f-e3b5e2844d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6817","nonce":1953907706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6819","nonce":1953907706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6825","nonce":1953907706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6821","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6822","nonce":1953907706}]},"public_addr":"172.21.15.195:6817/1953907706","cluster_addr":"172.21.15.195:6819/1953907706","heartbeat_back_addr":"172.21.15.195:6825/1953907706","heartbeat_front_addr":"172.21.15.195:6822/1953907706","state":["exists","up"]},{"osd":6,"uuid":"af8b34fd-927b-4860-971b-9fddf699a0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6801","nonce":4106887608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6805","nonce":4106887608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6812","nonce":4106887608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6808","nonce":4106887608}]},"public_addr":"172.21.15.195:6801/4106887608","cluster_addr":"172.21.15.195:6805/4106887608","heartbeat_back_addr":"172.21.15.195:6812/4106887608","heartbeat_front_addr":"172.21.15.195:6808/4106887608","state":["exists","up"]},{"osd":7,"uuid":"19a39645-8664-463b-bae3-d7aa1026e5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6823","nonce":3378300314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6827","nonce":3378300314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6831","nonce":3378300314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6829","nonce":3378300314}]},"public_addr":"172.21.15.195:6823/3378300314","cluster_addr":"172.21.15.195:6827/3378300314","heartbeat_back_addr":"172.21.15.195:6831/3378300314","heartbeat_front_addr":"172.21.15.195:6829/3378300314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:31.877950+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:30.243386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:30.409185+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:29.949919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T07:34:41.691 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 10 librados: watch_flush enter 2024-03-13T07:34:41.691 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 10 librados: watch_flush exit 2024-03-13T07:34:41.691 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 20 client.4246.objecter shutdown clearing up homeless session... 2024-03-13T07:34:41.691 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 10 client.4246.objecter successfully canceled tick 2024-03-13T07:34:41.691 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 1 -- 172.21.15.132:0/1209450039 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f0b0406d560 msgr2=0x7f0b0406fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:41.691 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 1 --2- 172.21.15.132:0/1209450039 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f0b0406d560 0x7f0b0406fa20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400b8d0 tx=0x7f0b14032000 comp rx=0 tx=0).stop 2024-03-13T07:34:41.692 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 10 monclient: shutdown 2024-03-13T07:34:41.692 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:41.692 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 1 -- 172.21.15.132:0/1209450039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2413f1e0 msgr2=0x7f0b24140910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:41.692 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 1 --2- 172.21.15.132:0/1209450039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2413f1e0 0x7f0b24140910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00c760 tx=0x7f0b1c00cc20 comp rx=0 tx=0).stop 2024-03-13T07:34:41.692 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 1 -- 172.21.15.132:0/1209450039 shutdown_connections 2024-03-13T07:34:41.692 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 1 --2- 172.21.15.132:0/1209450039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b2413f1e0 0x7f0b24140910 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:41.692 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 1 --2- 172.21.15.132:0/1209450039 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f0b0406d560 0x7f0b0406fa20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:41.692 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 1 --2- 172.21.15.132:0/1209450039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b2406fb00 0x7f0b241383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:41.692 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 1 --2- 172.21.15.132:0/1209450039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b2406f160 0x7f0b24137e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:41.692 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 1 -- 172.21.15.132:0/1209450039 >> 172.21.15.132:0/1209450039 conn(0x7f0b240b3df0 msgr2=0x7f0b240aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:41.692 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 1 -- 172.21.15.132:0/1209450039 shutdown_connections 2024-03-13T07:34:41.692 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 1 -- 172.21.15.132:0/1209450039 wait complete. 2024-03-13T07:34:41.692 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:41.691+0000 7f0b288c0640 1 librados: shutdown 2024-03-13T07:34:41.702 DEBUG:teuthology.misc:4 of 8 OSDs are up 2024-03-13T07:34:41.714 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:41.682+0000 5dc3480 -1 bdev(0x1865c9b0 /var/lib/ceph/osd/ceph-1/block) aio_submit retries 2 2024-03-13T07:34:41.754 INFO:tasks.ceph.osd.2.smithi132.stderr:2024-03-13T07:34:41.239+0000 5dc3480 -1 bdev(0x17e75d10 /var/lib/ceph/osd/ceph-2/block) aio_submit retries 2 2024-03-13T07:34:41.957 INFO:tasks.ceph.osd.2.smithi132.stderr:2024-03-13T07:34:41.755+0000 5dc3480 -1 bdev(0x17e75d10 /var/lib/ceph/osd/ceph-2/block) aio_submit retries 1 2024-03-13T07:34:41.958 INFO:tasks.ceph.osd.2.smithi132.stderr:2024-03-13T07:34:41.898+0000 5dc3480 -1 bdev(0x17e75d10 /var/lib/ceph/osd/ceph-2/block) aio_submit retries 1 2024-03-13T07:34:42.811 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:42.798+0000 b47a640 -1 osd.1 0 waiting for initial osdmap 2024-03-13T07:34:42.843 INFO:tasks.ceph.osd.0.smithi132.stderr:2024-03-13T07:34:42.840+0000 bc8e640 -1 osd.0 0 waiting for initial osdmap 2024-03-13T07:34:43.040 INFO:tasks.ceph.osd.2.smithi132.stderr:2024-03-13T07:34:43.035+0000 bc8e640 -1 osd.2 0 waiting for initial osdmap 2024-03-13T07:34:43.328 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:34:43.313+0000 1425d640 -1 osd.1 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T07:34:43.447 INFO:tasks.ceph.osd.0.smithi132.stderr:2024-03-13T07:34:43.434+0000 1425d640 -1 osd.0 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T07:34:43.562 INFO:tasks.ceph.osd.2.smithi132.stderr:2024-03-13T07:34:43.549+0000 1425d640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T07:34:45.422 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:01:09.313 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:34:45.486 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:01:09.376 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:34:45.516 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:01:09.386 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:34:45.530 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:01:09.421 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:34:45.574 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:01:09.445 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:34:45.591 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:01:09.482 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:34:45.745 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:01:09.635 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:34:45.819 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:01:09.690 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:34:45.851 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:01:09.741 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:34:48.509 DEBUG:teuthology.orchestra.run.smithi132:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-13T07:34:48.598 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.598+0000 7f0464d4a640 10 monclient: get_monmap_and_config 2024-03-13T07:34:48.599 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.598+0000 7f0464d4a640 10 monclient: build_initial_monmap 2024-03-13T07:34:48.599 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.598+0000 7f0464d4a640 10 monclient: monmap: 2024-03-13T07:34:48.599 INFO:teuthology.misc.health.smithi132.stderr:epoch 0 2024-03-13T07:34:48.599 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:48.599 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:34:48.599931+0000 2024-03-13T07:34:48.599 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:34:48.599931+0000 2024-03-13T07:34:48.599 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:48.599 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:48.599 INFO:teuthology.misc.health.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:48.600 INFO:teuthology.misc.health.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:48.600 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:48.600 INFO:teuthology.misc.health.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:48.600 INFO:teuthology.misc.health.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:48.600 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:48.602 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.598+0000 7f0464d4a640 1 Processor -- start 2024-03-13T07:34:48.602 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.598+0000 7f0464d4a640 1 -- start start 2024-03-13T07:34:48.602 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.598+0000 7f0464d4a640 10 monclient: init 2024-03-13T07:34:48.603 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.599+0000 7f0464d4a640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:48.603 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.599+0000 7f0464d4a640 10 monclient: _add_conns ranks=[4,3,1,0,2] 2024-03-13T07:34:48.603 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.599+0000 7f0464d4a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f04600b5180 0x7f04600a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:48.603 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.599+0000 7f0464d4a640 10 monclient(hunting): picked mon.noname-c con 0x7f04600b5180 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:48.603 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.599+0000 7f0464d4a640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f04600aa210 0x7f0460145ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:48.603 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.599+0000 7f0464d4a640 10 monclient(hunting): picked mon.noname-a con 0x7f04600aa210 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:48.603 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.599+0000 7f0464d4a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0460146020 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:48.603 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.599+0000 7f0464d4a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04600ab350 con 0x7f0460146020 2024-03-13T07:34:48.603 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.599+0000 7f0464d4a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:48.603 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.599+0000 7f0464d4a640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04600b5560 con 0x7f04600aa210 2024-03-13T07:34:48.603 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.599+0000 7f0464d4a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:48.603 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.599+0000 7f0464d4a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04600ab630 con 0x7f04600b5180 2024-03-13T07:34:48.603 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.599+0000 7f0464d4a640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:48.603 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.599+0000 7f0464d4a640 10 monclient(hunting): authenticate will time out at 1316.009003s 2024-03-13T07:34:48.604 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.600+0000 7f0457fff640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f04600aa210 0x7f0460145ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:48.604 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.600+0000 7f045e59c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f04600b5180 0x7f04600a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:48.604 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.601+0000 7f045e59c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f04600b5180 0x7f04600a96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:48696/0 (socket says 172.21.15.132:48696) 2024-03-13T07:34:48.604 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.601+0000 7f045e59c640 1 -- 172.21.15.132:0/1628595197 learned_addr learned my addr 172.21.15.132:0/1628595197 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:48.604 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.601+0000 7f045e59c640 10 monclient(hunting): get_auth_request con 0x7f04600b5180 auth_method 0 2024-03-13T07:34:48.604 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.601+0000 7f045e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:48.604 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.601+0000 7f045e59c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:48.604 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.601+0000 7f045e59c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:48.604 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.601+0000 7f0457fff640 10 monclient(hunting): get_auth_request con 0x7f04600aa210 auth_method 0 2024-03-13T07:34:48.604 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.601+0000 7f0457fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:48.604 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.601+0000 7f0457fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:48.604 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.601+0000 7f0457fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:48.604 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.601+0000 7f0457fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:48.604 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.601+0000 7f0457fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:48.605 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.601+0000 7f0457fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:48.605 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.602+0000 7f045e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:48.605 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.602+0000 7f045e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:48.605 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.602+0000 7f045e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:48.605 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.603+0000 7f045e59c640 10 monclient(hunting): handle_auth_done global_id 4249 payload 274 2024-03-13T07:34:48.606 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.603+0000 7f045e59c640 1 -- 172.21.15.132:0/1628595197 >> v2:172.21.15.132:3300/0 conn(0x7f04600aa210 msgr2=0x7f0460145ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:48.606 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.603+0000 7f045e59c640 1 --2- 172.21.15.132:0/1628595197 >> v2:172.21.15.132:3300/0 conn(0x7f04600aa210 0x7f0460145ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:48.606 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.603+0000 7f045e59c640 1 -- 172.21.15.132:0/1628595197 >> v1:172.21.15.195:6789/0 conn(0x7f0460146020 legacy=0x7f046013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:48.606 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.603+0000 7f045e59c640 10 monclient: _finish_hunting 0 2024-03-13T07:34:48.606 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.603+0000 7f045e59c640 1 monclient: found mon.noname-c 2024-03-13T07:34:48.606 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.603+0000 7f045e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:48.606 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.603+0000 7f045e59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T07:34:48.606 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.603+0000 7f045e59c640 1 -- 172.21.15.132:0/1628595197 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04600b47d0 con 0x7f04600b5180 2024-03-13T07:34:48.607 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.607+0000 7f045e59c640 1 --2- 172.21.15.132:0/1628595197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04600b5180 0x7f04600a96d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0450009870 tx=0x7f045002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e38f878b39dd0abc server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:48.613 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.611+0000 7f045dd9b640 1 -- 172.21.15.132:0/1628595197 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450035020 con 0x7f04600b5180 2024-03-13T07:34:48.613 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.611+0000 7f045dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:48.613 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.611+0000 7f045dd9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:48.613 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.611+0000 7f045dd9b640 10 monclient: dump: 2024-03-13T07:34:48.613 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:48.613 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:48.613 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:48.613 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:48.613 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:48.613 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:48.614 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:48.614 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:48.614 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:48.614 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:48.614 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.611+0000 7f045dd9b640 10 monclient: _finish_auth 0 2024-03-13T07:34:48.614 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.611+0000 7f045dd9b640 10 monclient: _check_auth_tickets 2024-03-13T07:34:48.614 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.611+0000 7f045dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:48.614 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.611+0000 7f045dd9b640 1 -- 172.21.15.132:0/1628595197 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0450031870 con 0x7f04600b5180 2024-03-13T07:34:48.614 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.611+0000 7f045dd9b640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:48.614 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.611+0000 7f045dd9b640 1 -- 172.21.15.132:0/1628595197 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450031c40 con 0x7f04600b5180 2024-03-13T07:34:48.614 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.611+0000 7f045dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:48.614 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.611+0000 7f045dd9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:48.614 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.611+0000 7f045dd9b640 10 monclient: dump: 2024-03-13T07:34:48.615 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:48.615 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:48.615 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:48.615 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:48.615 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:48.615 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:48.615 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:48.615 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:48.615 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:48.615 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:48.615 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.611+0000 7f0464d4a640 5 monclient: authenticate success, global_id 4249 2024-03-13T07:34:48.615 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.611+0000 7f0464d4a640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:48.615 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.611+0000 7f0464d4a640 4 set_mon_vals no callback set 2024-03-13T07:34:48.615 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.612+0000 7f0464d4a640 10 monclient: shutdown 2024-03-13T07:34:48.616 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.612+0000 7f0464d4a640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:48.616 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.612+0000 7f0464d4a640 1 -- 172.21.15.132:0/1628595197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04600b5180 msgr2=0x7f04600a96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:48.616 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.612+0000 7f0464d4a640 1 --2- 172.21.15.132:0/1628595197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04600b5180 0x7f04600a96d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0450009870 tx=0x7f045002b910 comp rx=0 tx=0).stop 2024-03-13T07:34:48.616 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.612+0000 7f0464d4a640 1 -- 172.21.15.132:0/1628595197 shutdown_connections 2024-03-13T07:34:48.616 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.612+0000 7f0464d4a640 1 --2- 172.21.15.132:0/1628595197 >> v2:172.21.15.132:3300/0 conn(0x7f04600aa210 0x7f0460145ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:48.616 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.612+0000 7f0464d4a640 1 --2- 172.21.15.132:0/1628595197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04600b5180 0x7f04600a96d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:48.616 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.612+0000 7f0464d4a640 1 -- 172.21.15.132:0/1628595197 >> 172.21.15.132:0/1628595197 conn(0x7f0460072750 msgr2=0x7f0460072b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:48.616 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.612+0000 7f0464d4a640 1 -- 172.21.15.132:0/1628595197 shutdown_connections 2024-03-13T07:34:48.616 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.612+0000 7f0464d4a640 1 -- 172.21.15.132:0/1628595197 wait complete. 2024-03-13T07:34:48.620 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.613+0000 7f0464d4a640 10 monclient: build_initial_monmap 2024-03-13T07:34:48.620 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.613+0000 7f0464d4a640 10 monclient: monmap: 2024-03-13T07:34:48.620 INFO:teuthology.misc.health.smithi132.stderr:epoch 0 2024-03-13T07:34:48.620 INFO:teuthology.misc.health.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:48.620 INFO:teuthology.misc.health.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:48.620 INFO:teuthology.misc.health.smithi132.stderr:created 0.000000 2024-03-13T07:34:48.620 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:48.621 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:48.621 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:48.621 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:48.621 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:48.621 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:48.621 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.613+0000 7f0464d4a640 1 librados: starting msgr at 2024-03-13T07:34:48.621 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.613+0000 7f0464d4a640 1 librados: starting objecter 2024-03-13T07:34:48.621 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.613+0000 7f0464d4a640 1 Processor -- start 2024-03-13T07:34:48.621 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.613+0000 7f0464d4a640 1 -- start start 2024-03-13T07:34:48.621 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.613+0000 7f0464d4a640 1 librados: setting wanted keys 2024-03-13T07:34:48.621 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.613+0000 7f0464d4a640 1 librados: calling monclient init 2024-03-13T07:34:48.621 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.613+0000 7f0464d4a640 10 monclient: init 2024-03-13T07:34:48.621 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f0464d4a640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:48.622 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f0464d4a640 10 monclient: _add_conns ranks=[0,1,2] 2024-03-13T07:34:48.622 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f0464d4a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04600aa210 0x7f04601405d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:48.622 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f0464d4a640 10 monclient(hunting): picked mon.noname-a con 0x7f04600aa210 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:48.622 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f0464d4a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04600b5180 0x7f0460135360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:48.622 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f0464d4a640 10 monclient(hunting): picked mon.noname-b con 0x7f04600b5180 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:48.622 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f0464d4a640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0460146020 0x7f04601d82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:48.622 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f0464d4a640 10 monclient(hunting): picked mon.noname-c con 0x7f0460146020 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:48.622 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f0464d4a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:48.622 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f0464d4a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046013fd60 con 0x7f04600aa210 2024-03-13T07:34:48.622 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f0464d4a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:48.622 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f0464d4a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046013fbe0 con 0x7f04600b5180 2024-03-13T07:34:48.622 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f0464d4a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:48.623 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f0464d4a640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046013fee0 con 0x7f0460146020 2024-03-13T07:34:48.623 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f0464d4a640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:48.623 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f0464d4a640 10 monclient(hunting): authenticate will time out at 1316.023300s 2024-03-13T07:34:48.623 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f045ed9d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0460146020 0x7f04601d82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:48.623 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.614+0000 7f045e59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04600aa210 0x7f04601405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:48.623 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f0457fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04600b5180 0x7f0460135360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:48.623 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f045ed9d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0460146020 0x7f04601d82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:42568/0 (socket says 172.21.15.132:42568) 2024-03-13T07:34:48.623 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f045ed9d640 1 -- 172.21.15.132:0/2520098617 learned_addr learned my addr 172.21.15.132:0/2520098617 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:48.623 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f045ed9d640 10 monclient(hunting): get_auth_request con 0x7f0460146020 auth_method 0 2024-03-13T07:34:48.623 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f045ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:48.623 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f045ed9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:48.623 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f045ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:48.623 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f0457fff640 10 monclient(hunting): get_auth_request con 0x7f04600b5180 auth_method 0 2024-03-13T07:34:48.623 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f0457fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:48.624 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f0457fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:48.624 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f0457fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:48.624 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f045e59c640 10 monclient(hunting): get_auth_request con 0x7f04600aa210 auth_method 0 2024-03-13T07:34:48.624 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f045e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:48.624 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f045e59c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:48.624 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f045e59c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:48.624 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f0457fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:48.624 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f0457fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:48.624 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f0457fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:48.624 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f045ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:48.624 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f045ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:48.624 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f045ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:48.625 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f045e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:48.625 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f045e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:48.625 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.615+0000 7f045e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:48.625 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.618+0000 7f0457fff640 10 monclient(hunting): handle_auth_done global_id 4252 payload 824 2024-03-13T07:34:48.625 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.618+0000 7f0457fff640 1 -- 172.21.15.132:0/2520098617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0460146020 msgr2=0x7f04601d82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:48.625 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.618+0000 7f0457fff640 1 --2- 172.21.15.132:0/2520098617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0460146020 0x7f04601d82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:48.625 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.618+0000 7f0457fff640 1 -- 172.21.15.132:0/2520098617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04600aa210 msgr2=0x7f04601405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:48.625 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.618+0000 7f0457fff640 1 --2- 172.21.15.132:0/2520098617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04600aa210 0x7f04601405d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:48.625 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.618+0000 7f0457fff640 10 monclient: _finish_hunting 0 2024-03-13T07:34:48.625 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.618+0000 7f0457fff640 1 monclient: found mon.noname-b 2024-03-13T07:34:48.625 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.618+0000 7f0457fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:48.625 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.618+0000 7f0457fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:48.625 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.618+0000 7f0457fff640 1 -- 172.21.15.132:0/2520098617 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04601432f0 con 0x7f04600b5180 2024-03-13T07:34:48.627 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.620+0000 7f0457fff640 1 --2- 172.21.15.132:0/2520098617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04600b5180 0x7f0460135360 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f044800e850 tx=0x7f044800ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:48.627 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.620+0000 7f0456ffd640 10 client.?.objecter ms_handle_connect 0x7f04600b5180 2024-03-13T07:34:48.627 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.620+0000 7f0456ffd640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:48.627 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0456ffd640 1 -- 172.21.15.132:0/2520098617 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0448017070 con 0x7f04600b5180 2024-03-13T07:34:48.627 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0456ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:48.628 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0456ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:48.628 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0456ffd640 10 monclient: dump: 2024-03-13T07:34:48.628 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:48.628 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:48.628 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:48.628 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:48.628 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:48.628 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:48.628 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:48.628 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:48.628 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:48.628 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:48.628 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0456ffd640 10 monclient: _finish_auth 0 2024-03-13T07:34:48.629 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0456ffd640 10 monclient: _check_auth_tickets 2024-03-13T07:34:48.629 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0456ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:48.629 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0456ffd640 1 -- 172.21.15.132:0/2520098617 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044800c940 con 0x7f04600b5180 2024-03-13T07:34:48.629 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0464d4a640 5 monclient: authenticate success, global_id 4252 2024-03-13T07:34:48.629 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0456ffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:48.629 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0464d4a640 10 monclient: _renew_subs 2024-03-13T07:34:48.629 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0456ffd640 1 -- 172.21.15.132:0/2520098617 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044800cd10 con 0x7f04600b5180 2024-03-13T07:34:48.629 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0464d4a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:48.629 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0464d4a640 1 -- 172.21.15.132:0/2520098617 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0460142a70 con 0x7f04600b5180 2024-03-13T07:34:48.629 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0456ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:48.629 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0456ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:48.629 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0456ffd640 10 monclient: dump: 2024-03-13T07:34:48.629 INFO:teuthology.misc.health.smithi132.stderr:epoch 1 2024-03-13T07:34:48.629 INFO:teuthology.misc.health.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:48.630 INFO:teuthology.misc.health.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:48.630 INFO:teuthology.misc.health.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:48.630 INFO:teuthology.misc.health.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:48.630 INFO:teuthology.misc.health.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:48.630 INFO:teuthology.misc.health.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:48.630 INFO:teuthology.misc.health.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:48.630 INFO:teuthology.misc.health.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:48.630 INFO:teuthology.misc.health.smithi132.stderr: 2024-03-13T07:34:48.630 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f045d59a640 4 set_mon_vals no callback set 2024-03-13T07:34:48.630 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0464d4a640 10 client.4252.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:48.630 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0464d4a640 10 monclient: _renew_subs 2024-03-13T07:34:48.630 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0464d4a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:48.630 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0464d4a640 1 -- 172.21.15.132:0/2520098617 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0460142e30 con 0x7f04600b5180 2024-03-13T07:34:48.631 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.624+0000 7f0464d4a640 1 librados: init done 2024-03-13T07:34:48.631 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.625+0000 7f0464d4a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:48.631 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.625+0000 7f0464d4a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:48.631 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.625+0000 7f0464d4a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:48.631 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.625+0000 7f0464d4a640 1 -- 172.21.15.132:0/2520098617 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04600b3c00 con 0x7f04600b5180 2024-03-13T07:34:48.648 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.645+0000 7f0456ffd640 1 -- 172.21.15.132:0/2520098617 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f044801a070 con 0x7f04600b5180 2024-03-13T07:34:48.648 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.645+0000 7f0456ffd640 1 --2- 172.21.15.132:0/2520098617 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f044006d560 0x7f044006fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:48.648 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.646+0000 7f045e59c640 1 --2- 172.21.15.132:0/2520098617 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f044006d560 0x7f044006fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:48.648 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.646+0000 7f045e59c640 10 monclient: get_auth_request con 0x7f044006d560 auth_method 0 2024-03-13T07:34:48.648 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.646+0000 7f0456ffd640 10 client.4252.objecter ms_handle_connect 0x7f044006d560 2024-03-13T07:34:48.648 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.646+0000 7f045e59c640 1 --2- 172.21.15.132:0/2520098617 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f044006d560 0x7f044006fa20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0450036000 tx=0x7f045003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:48.648 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.647+0000 7f0456ffd640 1 -- 172.21.15.132:0/2520098617 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f044805f8e0 con 0x7f04600b5180 2024-03-13T07:34:48.648 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.647+0000 7f0456ffd640 10 client.4252.objecter ms_dispatch 0x7f0460000b90 osd_map(22..22 src has 1..22) v4 2024-03-13T07:34:48.648 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.647+0000 7f0456ffd640 3 client.4252.objecter handle_osd_map got epochs [22,22] > 0 2024-03-13T07:34:48.648 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.647+0000 7f0456ffd640 3 client.4252.objecter handle_osd_map decoding full epoch 22 2024-03-13T07:34:48.649 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.647+0000 7f0456ffd640 20 client.4252.objecter dump_active .. 0 homeless 2024-03-13T07:34:48.882 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.881+0000 7f0456ffd640 1 -- 172.21.15.132:0/2520098617 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f04600b3c00 con 0x7f04600b5180 2024-03-13T07:34:48.883 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.881+0000 7f0456ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:48.883 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:48.881+0000 7f0456ffd640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:49.058 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.057+0000 7f0464d4a640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:49.058 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.057+0000 7f0464d4a640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:49.058 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.057+0000 7f0464d4a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:49.059 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.057+0000 7f0464d4a640 1 -- 172.21.15.132:0/2520098617 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04601430f0 con 0x7f04600b5180 2024-03-13T07:34:49.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.083+0000 7f0456ffd640 1 -- 172.21.15.132:0/2520098617 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+12909 (secure 0 0 0) 0x7f0460142a70 con 0x7f04600b5180 2024-03-13T07:34:49.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.083+0000 7f0456ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:49.084 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.083+0000 7f0456ffd640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:49.084 INFO:teuthology.misc.health.smithi132.stdout: 2024-03-13T07:34:49.084 INFO:teuthology.misc.health.smithi132.stdout:{"epoch":22,"fsid":"74e4bc2f-6a5e-4423-a8c8-807970a2f96c","created":"2024-03-13T07:33:47.494293+0000","modified":"2024-03-13T07:34:47.055546+0000","last_up_change":"2024-03-13T07:34:44.296009+0000","last_in_change":"2024-03-13T07:33:53.171597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-13T07:34:33.504752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0edb216c-bc1e-4865-aafe-218611f3eb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":590522438},{"type":"v1","addr":"172.21.15.132:6819","nonce":590522438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":590522438},{"type":"v1","addr":"172.21.15.132:6821","nonce":590522438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":590522438},{"type":"v1","addr":"172.21.15.132:6825","nonce":590522438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":590522438},{"type":"v1","addr":"172.21.15.132:6823","nonce":590522438}]},"public_addr":"172.21.15.132:6819/590522438","cluster_addr":"172.21.15.132:6821/590522438","heartbeat_back_addr":"172.21.15.132:6825/590522438","heartbeat_front_addr":"172.21.15.132:6823/590522438","state":["exists","up"]},{"osd":1,"uuid":"6be0ae6b-951e-4279-ba0a-9b05b6e6e85e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1071068206},{"type":"v1","addr":"172.21.15.132:6803","nonce":1071068206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1071068206},{"type":"v1","addr":"172.21.15.132:6805","nonce":1071068206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1071068206},{"type":"v1","addr":"172.21.15.132:6809","nonce":1071068206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1071068206},{"type":"v1","addr":"172.21.15.132:6807","nonce":1071068206}]},"public_addr":"172.21.15.132:6803/1071068206","cluster_addr":"172.21.15.132:6805/1071068206","heartbeat_back_addr":"172.21.15.132:6809/1071068206","heartbeat_front_addr":"172.21.15.132:6807/1071068206","state":["exists","up"]},{"osd":2,"uuid":"18008b7f-cfe7-4450-a3cf-aab63ebbe99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":958254560},{"type":"v1","addr":"172.21.15.132:6827","nonce":958254560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":958254560},{"type":"v1","addr":"172.21.15.132:6829","nonce":958254560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":958254560},{"type":"v1","addr":"172.21.15.132:6833","nonce":958254560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":958254560},{"type":"v1","addr":"172.21.15.132:6831","nonce":958254560}]},"public_addr":"172.21.15.132:6827/958254560","cluster_addr":"172.21.15.132:6829/958254560","heartbeat_back_addr":"172.21.15.132:6833/958254560","heartbeat_front_addr":"172.21.15.132:6831/958254560","state":["exists","up"]},{"osd":3,"uuid":"411b6066-4b9f-40e7-b92d-4ba914eab63e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6811","nonce":1770057764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6813","nonce":1770057764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6817","nonce":1770057764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6815","nonce":1770057764}]},"public_addr":"172.21.15.132:6811/1770057764","cluster_addr":"172.21.15.132:6813/1770057764","heartbeat_back_addr":"172.21.15.132:6817/1770057764","heartbeat_front_addr":"172.21.15.132:6815/1770057764","state":["exists","up"]},{"osd":4,"uuid":"1a719246-ae04-42b0-bd9c-fc318dc57ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":699638467},{"type":"v1","addr":"172.21.15.195:6803","nonce":699638467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":699638467},{"type":"v1","addr":"172.21.15.195:6809","nonce":699638467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":699638467},{"type":"v1","addr":"172.21.15.195:6815","nonce":699638467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6811","nonce":699638467},{"type":"v1","addr":"172.21.15.195:6813","nonce":699638467}]},"public_addr":"172.21.15.195:6803/699638467","cluster_addr":"172.21.15.195:6809/699638467","heartbeat_back_addr":"172.21.15.195:6815/699638467","heartbeat_front_addr":"172.21.15.195:6813/699638467","state":["exists","up"]},{"osd":5,"uuid":"9d9ee991-241b-4507-ba2f-e3b5e2844d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6817","nonce":1953907706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6819","nonce":1953907706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6825","nonce":1953907706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6821","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6822","nonce":1953907706}]},"public_addr":"172.21.15.195:6817/1953907706","cluster_addr":"172.21.15.195:6819/1953907706","heartbeat_back_addr":"172.21.15.195:6825/1953907706","heartbeat_front_addr":"172.21.15.195:6822/1953907706","state":["exists","up"]},{"osd":6,"uuid":"af8b34fd-927b-4860-971b-9fddf699a0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6801","nonce":4106887608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6805","nonce":4106887608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6812","nonce":4106887608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6808","nonce":4106887608}]},"public_addr":"172.21.15.195:6801/4106887608","cluster_addr":"172.21.15.195:6805/4106887608","heartbeat_back_addr":"172.21.15.195:6812/4106887608","heartbeat_front_addr":"172.21.15.195:6808/4106887608","state":["exists","up"]},{"osd":7,"uuid":"19a39645-8664-463b-bae3-d7aa1026e5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6823","nonce":3378300314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6827","nonce":3378300314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6831","nonce":3378300314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6829","nonce":3378300314}]},"public_addr":"172.21.15.195:6823/3378300314","cluster_addr":"172.21.15.195:6827/3378300314","heartbeat_back_addr":"172.21.15.195:6831/3378300314","heartbeat_front_addr":"172.21.15.195:6829/3378300314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:32.324299+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:31.786840+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:32.474925+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:31.877950+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:30.195463+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:30.243386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:30.409185+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:29.949919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T07:34:49.087 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 10 librados: watch_flush enter 2024-03-13T07:34:49.088 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 10 librados: watch_flush exit 2024-03-13T07:34:49.088 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 20 client.4252.objecter shutdown clearing up homeless session... 2024-03-13T07:34:49.088 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 10 client.4252.objecter successfully canceled tick 2024-03-13T07:34:49.088 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 1 -- 172.21.15.132:0/2520098617 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f044006d560 msgr2=0x7f044006fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:49.088 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 1 --2- 172.21.15.132:0/2520098617 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f044006d560 0x7f044006fa20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0450036000 tx=0x7f045003c040 comp rx=0 tx=0).stop 2024-03-13T07:34:49.088 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 10 monclient: shutdown 2024-03-13T07:34:49.088 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:49.089 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 1 -- 172.21.15.132:0/2520098617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04600b5180 msgr2=0x7f0460135360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:49.089 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 1 --2- 172.21.15.132:0/2520098617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04600b5180 0x7f0460135360 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f044800e850 tx=0x7f044800ed10 comp rx=0 tx=0).stop 2024-03-13T07:34:49.089 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 1 -- 172.21.15.132:0/2520098617 shutdown_connections 2024-03-13T07:34:49.089 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 1 --2- 172.21.15.132:0/2520098617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0460146020 0x7f04601d82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:49.089 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 1 --2- 172.21.15.132:0/2520098617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04600b5180 0x7f0460135360 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:49.089 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 1 --2- 172.21.15.132:0/2520098617 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f044006d560 0x7f044006fa20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:49.090 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 1 --2- 172.21.15.132:0/2520098617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04600aa210 0x7f04601405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:49.090 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 1 -- 172.21.15.132:0/2520098617 >> 172.21.15.132:0/2520098617 conn(0x7f0460072750 msgr2=0x7f046013ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:49.090 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 1 -- 172.21.15.132:0/2520098617 shutdown_connections 2024-03-13T07:34:49.090 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 1 -- 172.21.15.132:0/2520098617 wait complete. 2024-03-13T07:34:49.090 INFO:teuthology.misc.health.smithi132.stderr:2024-03-13T07:34:49.085+0000 7f0464d4a640 1 librados: shutdown 2024-03-13T07:34:49.096 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-13T07:34:49.096 INFO:tasks.ceph:Creating RBD pool 2024-03-13T07:34:49.096 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd pool create rbd 8 2024-03-13T07:34:49.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.201+0000 7fad18049640 10 monclient: get_monmap_and_config 2024-03-13T07:34:49.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.201+0000 7fad18049640 10 monclient: build_initial_monmap 2024-03-13T07:34:49.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.201+0000 7fad18049640 10 monclient: monmap: 2024-03-13T07:34:49.203 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:49.204 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:49.204 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:49.203170+0000 2024-03-13T07:34:49.204 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:49.203170+0000 2024-03-13T07:34:49.204 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:49.204 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:49.204 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:49.205 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:49.205 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:49.205 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:49.205 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:49.205 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:49.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.201+0000 7fad18049640 1 Processor -- start 2024-03-13T07:34:49.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.202+0000 7fad18049640 1 -- start start 2024-03-13T07:34:49.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.202+0000 7fad18049640 10 monclient: init 2024-03-13T07:34:49.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.202+0000 7fad18049640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:49.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.202+0000 7fad18049640 10 monclient: _add_conns ranks=[3,4,2,1,0] 2024-03-13T07:34:49.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.202+0000 7fad18049640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fad100b5310 0x7fad1013f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:49.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.202+0000 7fad18049640 10 monclient(hunting): picked mon.noname-a con 0x7fad100b5310 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:49.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.202+0000 7fad18049640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fad100afa20 0x7fad1013fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:49.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.202+0000 7fad18049640 10 monclient(hunting): picked mon.noname-c con 0x7fad100afa20 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:49.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.203+0000 7fad18049640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad100b1b00 0x7fad100b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:49.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.203+0000 7fad18049640 10 monclient(hunting): picked mon.noname-b con 0x7fad100b1b00 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:49.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.203+0000 7fad18049640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:49.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.203+0000 7fad18049640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad100b56f0 con 0x7fad100b5310 2024-03-13T07:34:49.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.203+0000 7fad18049640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:49.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.203+0000 7fad18049640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad100b16e0 con 0x7fad100afa20 2024-03-13T07:34:49.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.203+0000 7fad18049640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:49.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.203+0000 7fad18049640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad100afeb0 con 0x7fad100b1b00 2024-03-13T07:34:49.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.203+0000 7fad18049640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:49.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.203+0000 7fad18049640 10 monclient(hunting): authenticate will time out at 1316.612129s 2024-03-13T07:34:49.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.204+0000 7fad155e4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fad100afa20 0x7fad1013fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:49.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.204+0000 7fad155e4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fad100afa20 0x7fad1013fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:48724/0 (socket says 172.21.15.132:48724) 2024-03-13T07:34:49.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.204+0000 7fad155e4640 1 -- 172.21.15.132:0/4231396077 learned_addr learned my addr 172.21.15.132:0/4231396077 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:49.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.204+0000 7fad155e4640 10 monclient(hunting): get_auth_request con 0x7fad100afa20 auth_method 0 2024-03-13T07:34:49.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.204+0000 7fad155e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:49.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.204+0000 7fad155e4640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:49.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.204+0000 7fad155e4640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:49.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.204+0000 7fad165e6640 1 --2- 172.21.15.132:0/4231396077 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad100b1b00 0x7fad100b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:49.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.204+0000 7fad165e6640 10 monclient(hunting): get_auth_request con 0x7fad100b1b00 auth_method 0 2024-03-13T07:34:49.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.204+0000 7fad165e6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:49.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.204+0000 7fad165e6640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:49.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.204+0000 7fad165e6640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:49.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.204+0000 7fad155e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:49.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.204+0000 7fad155e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:49.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.204+0000 7fad155e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:49.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.205+0000 7fad15de5640 1 --2- 172.21.15.132:0/4231396077 >> v2:172.21.15.132:3300/0 conn(0x7fad100b5310 0x7fad1013f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:49.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.205+0000 7fad165e6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:49.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.205+0000 7fad165e6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:49.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.205+0000 7fad165e6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:49.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.206+0000 7fad15de5640 10 monclient(hunting): get_auth_request con 0x7fad100b5310 auth_method 0 2024-03-13T07:34:49.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.206+0000 7fad15de5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:49.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.206+0000 7fad15de5640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:49.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.206+0000 7fad15de5640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:49.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.206+0000 7fad155e4640 10 monclient(hunting): handle_auth_done global_id 4255 payload 274 2024-03-13T07:34:49.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.206+0000 7fad155e4640 1 -- 172.21.15.132:0/4231396077 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad100b1b00 msgr2=0x7fad100b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:49.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.206+0000 7fad155e4640 1 --2- 172.21.15.132:0/4231396077 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad100b1b00 0x7fad100b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:49.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.206+0000 7fad155e4640 1 -- 172.21.15.132:0/4231396077 >> v2:172.21.15.132:3300/0 conn(0x7fad100b5310 msgr2=0x7fad1013f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:49.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.206+0000 7fad155e4640 1 --2- 172.21.15.132:0/4231396077 >> v2:172.21.15.132:3300/0 conn(0x7fad100b5310 0x7fad1013f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:49.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.206+0000 7fad155e4640 10 monclient: _finish_hunting 0 2024-03-13T07:34:49.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.206+0000 7fad155e4640 1 monclient: found mon.noname-c 2024-03-13T07:34:49.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.206+0000 7fad155e4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:49.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.206+0000 7fad155e4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T07:34:49.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.206+0000 7fad155e4640 1 -- 172.21.15.132:0/4231396077 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad100b4820 con 0x7fad100afa20 2024-03-13T07:34:49.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.208+0000 7fad155e4640 1 --2- 172.21.15.132:0/4231396077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad100afa20 0x7fad1013fea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fad100b5c10 tx=0x7fad0002b7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=374997ab23e35616 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:49.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad14de3640 1 -- 172.21.15.132:0/4231396077 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00038070 con 0x7fad100afa20 2024-03-13T07:34:49.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad14de3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:49.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad14de3640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:49.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad14de3640 10 monclient: dump: 2024-03-13T07:34:49.219 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:49.219 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:49.219 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:49.219 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:49.219 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:49.219 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:49.219 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:49.220 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:49.220 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:49.220 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:49.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad14de3640 10 monclient: _finish_auth 0 2024-03-13T07:34:49.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad14de3640 10 monclient: _check_auth_tickets 2024-03-13T07:34:49.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad14de3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:49.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad14de3640 1 -- 172.21.15.132:0/4231396077 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0002f050 con 0x7fad100afa20 2024-03-13T07:34:49.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad18049640 5 monclient: authenticate success, global_id 4255 2024-03-13T07:34:49.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad18049640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:49.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad14de3640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:49.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad18049640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:49.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad18049640 4 set_mon_vals no callback set 2024-03-13T07:34:49.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad14de3640 1 -- 172.21.15.132:0/4231396077 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00031970 con 0x7fad100afa20 2024-03-13T07:34:49.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad14de3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:49.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad14de3640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:49.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.213+0000 7fad14de3640 10 monclient: dump: 2024-03-13T07:34:49.221 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:49.221 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:49.221 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:49.221 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:49.221 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:49.221 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:49.221 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:49.221 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:49.221 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:49.221 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:49.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.214+0000 7fad18049640 10 monclient: shutdown 2024-03-13T07:34:49.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.214+0000 7fad18049640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:49.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.214+0000 7fad18049640 1 -- 172.21.15.132:0/4231396077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad100afa20 msgr2=0x7fad1013fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:49.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.214+0000 7fad18049640 1 --2- 172.21.15.132:0/4231396077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad100afa20 0x7fad1013fea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fad100b5c10 tx=0x7fad0002b7f0 comp rx=0 tx=0).stop 2024-03-13T07:34:49.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.214+0000 7fad18049640 1 -- 172.21.15.132:0/4231396077 shutdown_connections 2024-03-13T07:34:49.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.214+0000 7fad18049640 1 --2- 172.21.15.132:0/4231396077 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad100b1b00 0x7fad100b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:49.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.214+0000 7fad18049640 1 --2- 172.21.15.132:0/4231396077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad100afa20 0x7fad1013fea0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:49.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.214+0000 7fad18049640 1 --2- 172.21.15.132:0/4231396077 >> v2:172.21.15.132:3300/0 conn(0x7fad100b5310 0x7fad1013f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:49.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.214+0000 7fad18049640 1 -- 172.21.15.132:0/4231396077 >> 172.21.15.132:0/4231396077 conn(0x7fad1006d2b0 msgr2=0x7fad1006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:49.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.214+0000 7fad18049640 1 -- 172.21.15.132:0/4231396077 shutdown_connections 2024-03-13T07:34:49.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.214+0000 7fad18049640 1 -- 172.21.15.132:0/4231396077 wait complete. 2024-03-13T07:34:49.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.215+0000 7fad18049640 10 monclient: build_initial_monmap 2024-03-13T07:34:49.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.215+0000 7fad18049640 10 monclient: monmap: 2024-03-13T07:34:49.222 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:49.222 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:49.222 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:49.222 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:49.223 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:49.223 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:49.223 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:49.223 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:49.223 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:49.223 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:49.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.215+0000 7fad18049640 1 librados: starting msgr at 2024-03-13T07:34:49.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.215+0000 7fad18049640 1 librados: starting objecter 2024-03-13T07:34:49.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.215+0000 7fad18049640 1 Processor -- start 2024-03-13T07:34:49.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.215+0000 7fad18049640 1 -- start start 2024-03-13T07:34:49.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.215+0000 7fad18049640 1 librados: setting wanted keys 2024-03-13T07:34:49.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.215+0000 7fad18049640 1 librados: calling monclient init 2024-03-13T07:34:49.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.215+0000 7fad18049640 10 monclient: init 2024-03-13T07:34:49.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.216+0000 7fad18049640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:49.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.216+0000 7fad18049640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T07:34:49.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.216+0000 7fad18049640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad100b1b00 0x7fad101ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:49.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.216+0000 7fad18049640 10 monclient(hunting): picked mon.noname-a con 0x7fad100b1b00 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:49.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.216+0000 7fad18049640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad100b5310 0x7fad100aa760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:49.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.216+0000 7fad18049640 10 monclient(hunting): picked mon.noname-c con 0x7fad100b5310 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:49.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.216+0000 7fad18049640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad101db100 0x7fad100ab450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:49.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.216+0000 7fad18049640 10 monclient(hunting): picked mon.noname-b con 0x7fad101db100 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:49.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.216+0000 7fad18049640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:49.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.216+0000 7fad18049640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad100b25c0 con 0x7fad100b1b00 2024-03-13T07:34:49.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.216+0000 7fad18049640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:49.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.216+0000 7fad18049640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad100b2440 con 0x7fad101db100 2024-03-13T07:34:49.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.216+0000 7fad18049640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:49.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.216+0000 7fad18049640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad100b2740 con 0x7fad100b5310 2024-03-13T07:34:49.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.216+0000 7fad18049640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:49.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.216+0000 7fad18049640 10 monclient(hunting): authenticate will time out at 1316.625404s 2024-03-13T07:34:49.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.217+0000 7fad155e4640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad100b5310 0x7fad100aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:49.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.217+0000 7fad155e4640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad100b5310 0x7fad100aa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:42572/0 (socket says 172.21.15.132:42572) 2024-03-13T07:34:49.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.217+0000 7fad155e4640 1 -- 172.21.15.132:0/3784520732 learned_addr learned my addr 172.21.15.132:0/3784520732 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:49.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.217+0000 7fad155e4640 10 monclient(hunting): get_auth_request con 0x7fad100b5310 auth_method 0 2024-03-13T07:34:49.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.217+0000 7fad155e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:49.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.217+0000 7fad155e4640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:49.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.217+0000 7fad155e4640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:49.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.218+0000 7fad155e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:49.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.218+0000 7fad155e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:49.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.218+0000 7fad155e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:49.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.220+0000 7fad155e4640 10 monclient(hunting): handle_auth_done global_id 4253 payload 824 2024-03-13T07:34:49.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.220+0000 7fad155e4640 1 -- 172.21.15.132:0/3784520732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad101db100 msgr2=0x7fad100ab450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:49.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.220+0000 7fad155e4640 1 --2- 172.21.15.132:0/3784520732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad101db100 0x7fad100ab450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:49.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.220+0000 7fad155e4640 1 -- 172.21.15.132:0/3784520732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad100b1b00 msgr2=0x7fad101ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:49.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.220+0000 7fad155e4640 1 --2- 172.21.15.132:0/3784520732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad100b1b00 0x7fad101ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:49.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.220+0000 7fad155e4640 10 monclient: _finish_hunting 0 2024-03-13T07:34:49.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.220+0000 7fad155e4640 1 monclient: found mon.noname-c 2024-03-13T07:34:49.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.220+0000 7fad155e4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:49.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.220+0000 7fad155e4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:34:49.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.220+0000 7fad155e4640 1 -- 172.21.15.132:0/3784520732 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad100aaca0 con 0x7fad100b5310 2024-03-13T07:34:49.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.223+0000 7fad155e4640 1 --2- 172.21.15.132:0/3784520732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad100b5310 0x7fad100aa760 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fad0002be00 tx=0x7fad000046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:49.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.223+0000 7fad067fc640 10 client.?.objecter ms_handle_connect 0x7fad100b5310 2024-03-13T07:34:49.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.223+0000 7fad067fc640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:49.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad067fc640 1 -- 172.21.15.132:0/3784520732 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00038040 con 0x7fad100b5310 2024-03-13T07:34:49.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:49.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad067fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:49.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad067fc640 10 monclient: dump: 2024-03-13T07:34:49.228 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:49.228 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:49.228 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:49.228 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:49.228 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:49.228 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:49.229 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad067fc640 10 monclient: _finish_auth 0 2024-03-13T07:34:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad067fc640 10 monclient: _check_auth_tickets 2024-03-13T07:34:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad067fc640 1 -- 172.21.15.132:0/3784520732 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad00043070 con 0x7fad100b5310 2024-03-13T07:34:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad067fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad067fc640 1 -- 172.21.15.132:0/3784520732 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00033880 con 0x7fad100b5310 2024-03-13T07:34:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad18049640 5 monclient: authenticate success, global_id 4253 2024-03-13T07:34:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad067fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad067fc640 10 monclient: dump: 2024-03-13T07:34:49.229 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:49.230 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad18049640 10 monclient: _renew_subs 2024-03-13T07:34:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad077fe640 4 set_mon_vals no callback set 2024-03-13T07:34:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad18049640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.226+0000 7fad18049640 1 -- 172.21.15.132:0/3784520732 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad100acae0 con 0x7fad100b5310 2024-03-13T07:34:49.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.227+0000 7fad18049640 10 client.4253.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:49.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.227+0000 7fad18049640 10 monclient: _renew_subs 2024-03-13T07:34:49.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.227+0000 7fad18049640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:49.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.227+0000 7fad18049640 1 -- 172.21.15.132:0/3784520732 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad100ab990 con 0x7fad100b5310 2024-03-13T07:34:49.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.227+0000 7fad18049640 1 librados: init done 2024-03-13T07:34:49.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.228+0000 7fad18049640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:49.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.228+0000 7fad18049640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:49.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.228+0000 7fad18049640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:49.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.228+0000 7fad18049640 1 -- 172.21.15.132:0/3784520732 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1004f000 con 0x7fad100b5310 2024-03-13T07:34:49.477 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.476+0000 7fad067fc640 1 -- 172.21.15.132:0/3784520732 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fad0003b070 con 0x7fad100b5310 2024-03-13T07:34:49.477 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.476+0000 7fad067fc640 1 --2- 172.21.15.132:0/3784520732 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7facf406d5b0 0x7facf406fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:49.477 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.476+0000 7fad15de5640 1 --2- 172.21.15.132:0/3784520732 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7facf406d5b0 0x7facf406fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:49.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.476+0000 7fad15de5640 10 monclient: get_auth_request con 0x7facf406d5b0 auth_method 0 2024-03-13T07:34:49.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.477+0000 7fad15de5640 1 --2- 172.21.15.132:0/3784520732 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7facf406d5b0 0x7facf406fa70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00b7f0 tx=0x7fad0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:49.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.477+0000 7fad067fc640 10 client.4253.objecter ms_handle_connect 0x7facf406d5b0 2024-03-13T07:34:49.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.489+0000 7fad067fc640 1 -- 172.21.15.132:0/3784520732 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fad00081370 con 0x7fad100b5310 2024-03-13T07:34:49.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.489+0000 7fad067fc640 10 client.4253.objecter ms_dispatch 0x7fad10000b90 osd_map(22..22 src has 1..22) v4 2024-03-13T07:34:49.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.489+0000 7fad067fc640 3 client.4253.objecter handle_osd_map got epochs [22,22] > 0 2024-03-13T07:34:49.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.489+0000 7fad067fc640 3 client.4253.objecter handle_osd_map decoding full epoch 22 2024-03-13T07:34:49.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.489+0000 7fad067fc640 20 client.4253.objecter dump_active .. 0 homeless 2024-03-13T07:34:49.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.489+0000 7fad067fc640 1 -- 172.21.15.132:0/3784520732 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fad0007b6a0 con 0x7fad100b5310 2024-03-13T07:34:49.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.489+0000 7fad067fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:49.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.489+0000 7fad067fc640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:49.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.687+0000 7fad18049640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-03-13T07:34:49.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.687+0000 7fad18049640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-03-13T07:34:49.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.687+0000 7fad18049640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:49.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:49.687+0000 7fad18049640 1 -- 172.21.15.132:0/3784520732 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fad100b8d80 con 0x7fad100b5310 2024-03-13T07:34:50.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.769+0000 7fad067fc640 1 -- 172.21.15.132:0/3784520732 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v23) v1 ==== 109+0+0 (secure 0 0 0) 0x7fad0007afa0 con 0x7fad100b5310 2024-03-13T07:34:50.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.769+0000 7fad067fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-03-13T07:34:50.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.769+0000 7fad067fc640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2024-03-13T07:34:50.771 INFO:teuthology.orchestra.run.smithi132.stderr:pool 'rbd' created 2024-03-13T07:34:50.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.771+0000 7fad18049640 10 librados: watch_flush enter 2024-03-13T07:34:50.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 10 librados: watch_flush exit 2024-03-13T07:34:50.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 20 client.4253.objecter shutdown clearing up homeless session... 2024-03-13T07:34:50.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 10 client.4253.objecter successfully canceled tick 2024-03-13T07:34:50.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 1 -- 172.21.15.132:0/3784520732 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7facf406d5b0 msgr2=0x7facf406fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:50.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 1 --2- 172.21.15.132:0/3784520732 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7facf406d5b0 0x7facf406fa70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00b7f0 tx=0x7fad0c008040 comp rx=0 tx=0).stop 2024-03-13T07:34:50.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 10 monclient: shutdown 2024-03-13T07:34:50.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:50.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 1 -- 172.21.15.132:0/3784520732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad100b5310 msgr2=0x7fad100aa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:50.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 1 --2- 172.21.15.132:0/3784520732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad100b5310 0x7fad100aa760 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fad0002be00 tx=0x7fad000046c0 comp rx=0 tx=0).stop 2024-03-13T07:34:50.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 1 -- 172.21.15.132:0/3784520732 shutdown_connections 2024-03-13T07:34:50.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 1 --2- 172.21.15.132:0/3784520732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad101db100 0x7fad100ab450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:50.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 1 --2- 172.21.15.132:0/3784520732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad100b5310 0x7fad100aa760 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:50.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 1 --2- 172.21.15.132:0/3784520732 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7facf406d5b0 0x7facf406fa70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:50.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 1 --2- 172.21.15.132:0/3784520732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad100b1b00 0x7fad101ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:50.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 1 -- 172.21.15.132:0/3784520732 >> 172.21.15.132:0/3784520732 conn(0x7fad1006d2b0 msgr2=0x7fad100b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:50.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 1 -- 172.21.15.132:0/3784520732 shutdown_connections 2024-03-13T07:34:50.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 1 -- 172.21.15.132:0/3784520732 wait complete. 2024-03-13T07:34:50.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.772+0000 7fad18049640 1 librados: shutdown 2024-03-13T07:34:50.784 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-03-13T07:34:50.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.887+0000 7fd4fdfc5640 10 monclient: get_monmap_and_config 2024-03-13T07:34:50.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.887+0000 7fd4fdfc5640 10 monclient: build_initial_monmap 2024-03-13T07:34:50.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.887+0000 7fd4fdfc5640 10 monclient: monmap: 2024-03-13T07:34:50.888 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:50.888 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:50.888 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:50.889360+0000 2024-03-13T07:34:50.888 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:50.889360+0000 2024-03-13T07:34:50.888 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:50.888 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:50.889 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:50.889 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:50.889 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:50.889 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:50.889 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:50.889 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:50.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.888+0000 7fd4fdfc5640 1 Processor -- start 2024-03-13T07:34:50.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.888+0000 7fd4fdfc5640 1 -- start start 2024-03-13T07:34:50.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.888+0000 7fd4fdfc5640 10 monclient: init 2024-03-13T07:34:50.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.888+0000 7fd4fdfc5640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:50.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.888+0000 7fd4fdfc5640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2024-03-13T07:34:50.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.889+0000 7fd4fdfc5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd4f80b52a0 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:50.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.889+0000 7fd4fdfc5640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd4f80b24b0 0x7fd4f8145c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:50.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.889+0000 7fd4fdfc5640 10 monclient(hunting): picked mon.noname-a con 0x7fd4f80b24b0 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:50.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.889+0000 7fd4fdfc5640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4f8140120 0x7fd4f81461c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:50.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.889+0000 7fd4fdfc5640 10 monclient(hunting): picked mon.noname-b con 0x7fd4f8140120 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:50.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.889+0000 7fd4fdfc5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f80b4790 con 0x7fd4f80b52a0 2024-03-13T07:34:50.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.889+0000 7fd4fdfc5640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:50.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.889+0000 7fd4fdfc5640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f80b5680 con 0x7fd4f80b24b0 2024-03-13T07:34:50.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.889+0000 7fd4fdfc5640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:50.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.889+0000 7fd4fdfc5640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f80aa4b0 con 0x7fd4f8140120 2024-03-13T07:34:50.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.889+0000 7fd4fdfc5640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:50.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.889+0000 7fd4fdfc5640 10 monclient(hunting): authenticate will time out at 1318.298183s 2024-03-13T07:34:50.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.889+0000 7fd4f7fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4f8140120 0x7fd4f81461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:50.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.889+0000 7fd4f77fe640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fd4f80b52a0 0x7fd4f813fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.132:47076/0 (socket says 172.21.15.132:47076) 2024-03-13T07:34:50.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.889+0000 7fd4f77fe640 1 -- 172.21.15.132:0/141529375 learned_addr learned my addr 172.21.15.132:0/141529375 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:34:50.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.889+0000 7fd4eeffd640 1 --2- 172.21.15.132:0/141529375 >> v2:172.21.15.132:3300/0 conn(0x7fd4f80b24b0 0x7fd4f8145c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:50.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.890+0000 7fd4f7fff640 10 monclient(hunting): get_auth_request con 0x7fd4f8140120 auth_method 0 2024-03-13T07:34:50.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.890+0000 7fd4f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:50.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.890+0000 7fd4f7fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:50.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.890+0000 7fd4f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:50.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.890+0000 7fd4eeffd640 10 monclient(hunting): get_auth_request con 0x7fd4f80b24b0 auth_method 0 2024-03-13T07:34:50.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.890+0000 7fd4eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:50.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.890+0000 7fd4eeffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:50.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.890+0000 7fd4eeffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:50.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.890+0000 7fd4f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:50.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.890+0000 7fd4f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:50.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.890+0000 7fd4f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:50.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.890+0000 7fd4eeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:50.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.890+0000 7fd4eeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:50.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.890+0000 7fd4eeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:50.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.892+0000 7fd4f6ffd640 1 -- 172.21.15.132:0/141529375 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3202910034 0 0) 0x7fd4f80b4790 con 0x7fd4f80b52a0 2024-03-13T07:34:50.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.892+0000 7fd4f7fff640 10 monclient(hunting): handle_auth_done global_id 4256 payload 274 2024-03-13T07:34:50.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.892+0000 7fd4f7fff640 1 -- 172.21.15.132:0/141529375 >> v2:172.21.15.132:3300/0 conn(0x7fd4f80b24b0 msgr2=0x7fd4f8145c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:50.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.892+0000 7fd4f7fff640 1 --2- 172.21.15.132:0/141529375 >> v2:172.21.15.132:3300/0 conn(0x7fd4f80b24b0 0x7fd4f8145c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:50.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.892+0000 7fd4f7fff640 1 -- 172.21.15.132:0/141529375 >> v1:172.21.15.195:6789/0 conn(0x7fd4f80b52a0 legacy=0x7fd4f813fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:50.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.892+0000 7fd4f7fff640 10 monclient: _finish_hunting 0 2024-03-13T07:34:50.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.892+0000 7fd4f7fff640 1 monclient: found mon.noname-b 2024-03-13T07:34:50.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.892+0000 7fd4f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:50.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.892+0000 7fd4f7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T07:34:50.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.892+0000 7fd4f7fff640 1 -- 172.21.15.132:0/141529375 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f8146700 con 0x7fd4f8140120 2024-03-13T07:34:50.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.892+0000 7fd4f6ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-03-13T07:34:50.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.894+0000 7fd4f7fff640 1 --2- 172.21.15.132:0/141529375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4f8140120 0x7fd4f81461c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009d90 tx=0x7fd4f002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af3a10d7dd6a275e server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:50.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4f6ffd640 1 -- 172.21.15.132:0/141529375 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0039070 con 0x7fd4f8140120 2024-03-13T07:34:50.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4f6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:50.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4f6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:50.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4f6ffd640 10 monclient: dump: 2024-03-13T07:34:50.899 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:50.899 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:50.899 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:50.899 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:50.899 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:50.899 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:50.900 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:50.900 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:50.900 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:50.900 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:50.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4f6ffd640 10 monclient: _finish_auth 0 2024-03-13T07:34:50.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4f6ffd640 10 monclient: _check_auth_tickets 2024-03-13T07:34:50.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:50.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4f6ffd640 1 -- 172.21.15.132:0/141529375 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f002cbc0 con 0x7fd4f8140120 2024-03-13T07:34:50.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4fdfc5640 5 monclient: authenticate success, global_id 4256 2024-03-13T07:34:50.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4fdfc5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:50.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4f6ffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:50.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4fdfc5640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:50.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4f6ffd640 1 -- 172.21.15.132:0/141529375 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0034440 con 0x7fd4f8140120 2024-03-13T07:34:50.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4fdfc5640 4 set_mon_vals no callback set 2024-03-13T07:34:50.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4f6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:50.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4f6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:50.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.898+0000 7fd4f6ffd640 10 monclient: dump: 2024-03-13T07:34:50.901 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:50.901 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:50.901 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:50.901 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:50.901 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:50.901 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:50.901 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:50.901 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:50.901 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:50.901 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:50.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.899+0000 7fd4fdfc5640 10 monclient: shutdown 2024-03-13T07:34:50.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.899+0000 7fd4fdfc5640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:50.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.899+0000 7fd4fdfc5640 1 -- 172.21.15.132:0/141529375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4f8140120 msgr2=0x7fd4f81461c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:50.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.899+0000 7fd4fdfc5640 1 --2- 172.21.15.132:0/141529375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4f8140120 0x7fd4f81461c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009d90 tx=0x7fd4f002be10 comp rx=0 tx=0).stop 2024-03-13T07:34:50.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.899+0000 7fd4fdfc5640 1 -- 172.21.15.132:0/141529375 shutdown_connections 2024-03-13T07:34:50.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.899+0000 7fd4fdfc5640 1 --2- 172.21.15.132:0/141529375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4f8140120 0x7fd4f81461c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:50.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.899+0000 7fd4fdfc5640 1 --2- 172.21.15.132:0/141529375 >> v2:172.21.15.132:3300/0 conn(0x7fd4f80b24b0 0x7fd4f8145c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:50.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.899+0000 7fd4fdfc5640 1 -- 172.21.15.132:0/141529375 >> 172.21.15.132:0/141529375 conn(0x7fd4f8066e20 msgr2=0x7fd4f8067230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:50.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.899+0000 7fd4fdfc5640 1 -- 172.21.15.132:0/141529375 shutdown_connections 2024-03-13T07:34:50.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.899+0000 7fd4fdfc5640 1 -- 172.21.15.132:0/141529375 wait complete. 2024-03-13T07:34:50.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.900+0000 7fd4fdfc5640 10 monclient: build_initial_monmap 2024-03-13T07:34:50.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.900+0000 7fd4fdfc5640 10 monclient: monmap: 2024-03-13T07:34:50.906 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:50.906 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:50.906 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:50.906 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:50.906 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:50.907 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:50.907 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:50.907 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:50.907 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:50.907 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:50.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.900+0000 7fd4fdfc5640 1 librados: starting msgr at 2024-03-13T07:34:50.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.900+0000 7fd4fdfc5640 1 librados: starting objecter 2024-03-13T07:34:50.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.900+0000 7fd4fdfc5640 1 Processor -- start 2024-03-13T07:34:50.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.900+0000 7fd4fdfc5640 1 -- start start 2024-03-13T07:34:50.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.900+0000 7fd4fdfc5640 1 librados: setting wanted keys 2024-03-13T07:34:50.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.900+0000 7fd4fdfc5640 1 librados: calling monclient init 2024-03-13T07:34:50.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.900+0000 7fd4fdfc5640 10 monclient: init 2024-03-13T07:34:50.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4fdfc5640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:50.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4fdfc5640 10 monclient: _add_conns ranks=[1,0,2] 2024-03-13T07:34:50.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4fdfc5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4f80b24b0 0x7fd4f8145390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:50.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4fdfc5640 10 monclient(hunting): picked mon.noname-b con 0x7fd4f80b24b0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:50.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4fdfc5640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f80b52a0 0x7fd4f81323f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:50.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4fdfc5640 10 monclient(hunting): picked mon.noname-a con 0x7fd4f80b52a0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:50.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4fdfc5640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4f8140120 0x7fd4f81dace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:50.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4fdfc5640 10 monclient(hunting): picked mon.noname-c con 0x7fd4f8140120 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:50.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4fdfc5640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:50.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4fdfc5640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f814cfb0 con 0x7fd4f80b52a0 2024-03-13T07:34:50.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4fdfc5640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:50.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4fdfc5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f80aa180 con 0x7fd4f80b24b0 2024-03-13T07:34:50.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4fdfc5640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:50.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4fdfc5640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f814d130 con 0x7fd4f8140120 2024-03-13T07:34:50.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4fdfc5640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:50.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4fdfc5640 10 monclient(hunting): authenticate will time out at 1318.310377s 2024-03-13T07:34:50.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4f77fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4f80b24b0 0x7fd4f8145390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:50.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4f7fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4f8140120 0x7fd4f81dace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:50.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.901+0000 7fd4eeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f80b52a0 0x7fd4f81323f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:50.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4f77fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4f80b24b0 0x7fd4f8145390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:48754/0 (socket says 172.21.15.132:48754) 2024-03-13T07:34:50.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4f77fe640 1 -- 172.21.15.132:0/470105797 learned_addr learned my addr 172.21.15.132:0/470105797 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:50.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4f77fe640 10 monclient(hunting): get_auth_request con 0x7fd4f80b24b0 auth_method 0 2024-03-13T07:34:50.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:50.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4f77fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:50.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:50.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4f7fff640 10 monclient(hunting): get_auth_request con 0x7fd4f8140120 auth_method 0 2024-03-13T07:34:50.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:50.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4f7fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:50.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:50.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4eeffd640 10 monclient(hunting): get_auth_request con 0x7fd4f80b52a0 auth_method 0 2024-03-13T07:34:50.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:50.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4eeffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:50.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4eeffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:50.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:50.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:50.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.902+0000 7fd4f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:50.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.903+0000 7fd4f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:50.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.903+0000 7fd4f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:50.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.903+0000 7fd4f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:50.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.903+0000 7fd4eeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:50.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.903+0000 7fd4eeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:50.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.903+0000 7fd4eeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:50.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.906+0000 7fd4eeffd640 10 monclient(hunting): handle_auth_done global_id 4254 payload 824 2024-03-13T07:34:50.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.906+0000 7fd4eeffd640 1 -- 172.21.15.132:0/470105797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4f8140120 msgr2=0x7fd4f81dace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:50.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.906+0000 7fd4eeffd640 1 --2- 172.21.15.132:0/470105797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4f8140120 0x7fd4f81dace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:50.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.906+0000 7fd4eeffd640 1 -- 172.21.15.132:0/470105797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4f80b24b0 msgr2=0x7fd4f8145390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:50.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.906+0000 7fd4eeffd640 1 --2- 172.21.15.132:0/470105797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4f80b24b0 0x7fd4f8145390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:50.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.906+0000 7fd4eeffd640 10 monclient: _finish_hunting 0 2024-03-13T07:34:50.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.906+0000 7fd4eeffd640 1 monclient: found mon.noname-a 2024-03-13T07:34:50.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.906+0000 7fd4eeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:50.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.906+0000 7fd4eeffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:50.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.906+0000 7fd4eeffd640 1 -- 172.21.15.132:0/470105797 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f8140580 con 0x7fd4f80b52a0 2024-03-13T07:34:50.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.906+0000 7fd4f77fe640 1 --2- 172.21.15.132:0/470105797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4f80b24b0 0x7fd4f8145390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-13T07:34:50.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.909+0000 7fd4eeffd640 1 --2- 172.21.15.132:0/470105797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f80b52a0 0x7fd4f81323f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000e820 tx=0x7fd4e000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:50.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.909+0000 7fd4f4ff9640 10 client.?.objecter ms_handle_connect 0x7fd4f80b52a0 2024-03-13T07:34:50.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.909+0000 7fd4f4ff9640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:50.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4f4ff9640 1 -- 172.21.15.132:0/470105797 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0017070 con 0x7fd4f80b52a0 2024-03-13T07:34:50.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4f4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:50.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4f4ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:50.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4f4ff9640 10 monclient: dump: 2024-03-13T07:34:50.914 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:50.914 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:50.914 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:50.914 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:50.914 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:50.914 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:50.914 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:50.914 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:50.914 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:50.915 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:50.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4f4ff9640 10 monclient: _finish_auth 0 2024-03-13T07:34:50.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4f4ff9640 10 monclient: _check_auth_tickets 2024-03-13T07:34:50.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4f4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:50.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4f4ff9640 1 -- 172.21.15.132:0/470105797 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e000c940 con 0x7fd4f80b52a0 2024-03-13T07:34:50.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4f4ff9640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:50.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4f4ff9640 1 -- 172.21.15.132:0/470105797 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000cd30 con 0x7fd4f80b52a0 2024-03-13T07:34:50.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4f4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:50.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4f4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:50.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4f4ff9640 10 monclient: dump: 2024-03-13T07:34:50.915 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:50.915 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:50.915 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:50.915 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:50.916 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:50.916 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:50.916 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:50.916 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:50.916 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:50.916 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:50.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4fdfc5640 5 monclient: authenticate success, global_id 4254 2024-03-13T07:34:50.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4fdfc5640 10 monclient: _renew_subs 2024-03-13T07:34:50.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4fdfc5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:50.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4fdfc5640 1 -- 172.21.15.132:0/470105797 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f8140880 con 0x7fd4f80b52a0 2024-03-13T07:34:50.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4f67fc640 4 set_mon_vals no callback set 2024-03-13T07:34:50.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4fdfc5640 10 client.4254.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:50.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4fdfc5640 10 monclient: _renew_subs 2024-03-13T07:34:50.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4fdfc5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:50.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.912+0000 7fd4fdfc5640 1 -- 172.21.15.132:0/470105797 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f8132b10 con 0x7fd4f80b52a0 2024-03-13T07:34:50.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.913+0000 7fd4fdfc5640 1 librados: init done 2024-03-13T07:34:50.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.913+0000 7fd4fdfc5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:50.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.914+0000 7fd4fdfc5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:50.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.914+0000 7fd4fdfc5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:50.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:50.914+0000 7fd4fdfc5640 1 -- 172.21.15.132:0/470105797 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f8140ab0 con 0x7fd4f80b52a0 2024-03-13T07:34:51.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.169+0000 7fd4f4ff9640 1 -- 172.21.15.132:0/470105797 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fd4e001a070 con 0x7fd4f80b52a0 2024-03-13T07:34:51.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.169+0000 7fd4f4ff9640 1 --2- 172.21.15.132:0/470105797 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fd4d806d5b0 0x7fd4d806fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:51.175 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.175+0000 7fd4f77fe640 1 --2- 172.21.15.132:0/470105797 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fd4d806d5b0 0x7fd4d806fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:51.175 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.175+0000 7fd4f77fe640 10 monclient: get_auth_request con 0x7fd4d806d5b0 auth_method 0 2024-03-13T07:34:51.176 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.175+0000 7fd4f77fe640 1 --2- 172.21.15.132:0/470105797 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fd4d806d5b0 0x7fd4d806fa70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8007b10 tx=0x7fd4e8007a60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:51.176 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.175+0000 7fd4f4ff9640 10 client.4254.objecter ms_handle_connect 0x7fd4d806d5b0 2024-03-13T07:34:51.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.184+0000 7fd4f4ff9640 1 -- 172.21.15.132:0/470105797 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5199+0+0 (secure 0 0 0) 0x7fd4e0057610 con 0x7fd4f80b52a0 2024-03-13T07:34:51.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.184+0000 7fd4f4ff9640 10 client.4254.objecter ms_dispatch 0x7fd4f8000b90 osd_map(23..23 src has 1..23) v4 2024-03-13T07:34:51.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.184+0000 7fd4f4ff9640 3 client.4254.objecter handle_osd_map got epochs [23,23] > 0 2024-03-13T07:34:51.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.184+0000 7fd4f4ff9640 3 client.4254.objecter handle_osd_map decoding full epoch 23 2024-03-13T07:34:51.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.184+0000 7fd4f4ff9640 20 client.4254.objecter dump_active .. 0 homeless 2024-03-13T07:34:51.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.184+0000 7fd4f4ff9640 1 -- 172.21.15.132:0/470105797 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fd4f8140ab0 con 0x7fd4f80b52a0 2024-03-13T07:34:51.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.184+0000 7fd4f4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:51.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.184+0000 7fd4f4ff9640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:51.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.425+0000 7fd4fdfc5640 10 start_mon_command cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2024-03-13T07:34:51.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.425+0000 7fd4fdfc5640 10 monclient: _send_command 2 [{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2024-03-13T07:34:51.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.425+0000 7fd4fdfc5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:51.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.425+0000 7fd4fdfc5640 1 -- 172.21.15.132:0/470105797 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd4f804f000 con 0x7fd4f80b52a0 2024-03-13T07:34:51.696 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.696+0000 7fd4f4ff9640 1 -- 172.21.15.132:0/470105797 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v24) v1 ==== 173+0+0 (secure 0 0 0) 0x7fd4e0059570 con 0x7fd4f80b52a0 2024-03-13T07:34:51.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.696+0000 7fd4f4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2024-03-13T07:34:51.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.696+0000 7fd4f4ff9640 10 monclient: _finish_command 2 = system:0 enabled application 'rbd' on pool 'rbd' 2024-03-13T07:34:51.697 INFO:teuthology.orchestra.run.smithi132.stderr:enabled application 'rbd' on pool 'rbd' 2024-03-13T07:34:51.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.697+0000 7fd4fdfc5640 10 librados: watch_flush enter 2024-03-13T07:34:51.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.697+0000 7fd4fdfc5640 10 librados: watch_flush exit 2024-03-13T07:34:51.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 20 client.4254.objecter shutdown clearing up homeless session... 2024-03-13T07:34:51.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 10 client.4254.objecter successfully canceled tick 2024-03-13T07:34:51.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 1 -- 172.21.15.132:0/470105797 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fd4d806d5b0 msgr2=0x7fd4d806fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:51.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 1 --2- 172.21.15.132:0/470105797 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fd4d806d5b0 0x7fd4d806fa70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8007b10 tx=0x7fd4e8007a60 comp rx=0 tx=0).stop 2024-03-13T07:34:51.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 10 monclient: shutdown 2024-03-13T07:34:51.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:51.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 1 -- 172.21.15.132:0/470105797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f80b52a0 msgr2=0x7fd4f81323f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:51.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 1 --2- 172.21.15.132:0/470105797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f80b52a0 0x7fd4f81323f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000e820 tx=0x7fd4e000ece0 comp rx=0 tx=0).stop 2024-03-13T07:34:51.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 1 -- 172.21.15.132:0/470105797 shutdown_connections 2024-03-13T07:34:51.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 1 --2- 172.21.15.132:0/470105797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4f8140120 0x7fd4f81dace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:51.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 1 --2- 172.21.15.132:0/470105797 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fd4d806d5b0 0x7fd4d806fa70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:51.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 1 --2- 172.21.15.132:0/470105797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f80b52a0 0x7fd4f81323f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:51.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 1 --2- 172.21.15.132:0/470105797 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4f80b24b0 0x7fd4f8145390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:51.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 1 -- 172.21.15.132:0/470105797 >> 172.21.15.132:0/470105797 conn(0x7fd4f8066e20 msgr2=0x7fd4f813f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:51.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 1 -- 172.21.15.132:0/470105797 shutdown_connections 2024-03-13T07:34:51.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 1 -- 172.21.15.132:0/470105797 wait complete. 2024-03-13T07:34:51.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.698+0000 7fd4fdfc5640 1 librados: shutdown 2024-03-13T07:34:51.711 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-13T07:34:51.711 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-13T07:34:51.712 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-13T07:34:51.712 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-13T07:34:51.712 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-13T07:34:51.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.822+0000 7f6d3ba71640 10 monclient: get_monmap_and_config 2024-03-13T07:34:51.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.822+0000 7f6d3ba71640 10 monclient: build_initial_monmap 2024-03-13T07:34:51.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.822+0000 7f6d3ba71640 10 monclient: monmap: 2024-03-13T07:34:51.823 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:51.824 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:51.824 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:51.824628+0000 2024-03-13T07:34:51.824 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:51.824628+0000 2024-03-13T07:34:51.824 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:51.824 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:51.824 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:51.824 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:51.824 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:51.824 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:51.824 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:51.824 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:51.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.823+0000 7f6d3ba71640 1 Processor -- start 2024-03-13T07:34:51.825 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.823+0000 7f6d3ba71640 1 -- start start 2024-03-13T07:34:51.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.823+0000 7f6d3ba71640 10 monclient: init 2024-03-13T07:34:51.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.824+0000 7f6d3ba71640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:51.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.824+0000 7f6d3ba71640 10 monclient: _add_conns ranks=[0,4,2,1,3] 2024-03-13T07:34:51.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.824+0000 7f6d3ba71640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6d340b1e70 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:51.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.824+0000 7f6d3ba71640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6d340b00f0 0x7f6d3413fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:51.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.824+0000 7f6d3ba71640 10 monclient(hunting): picked mon.noname-c con 0x7f6d340b00f0 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:51.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.824+0000 7f6d3ba71640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d34140400 0x7f6d34142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:51.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.824+0000 7f6d3ba71640 10 monclient(hunting): picked mon.noname-b con 0x7f6d34140400 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:51.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.824+0000 7f6d3ba71640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d340b1390 con 0x7f6d340b1e70 2024-03-13T07:34:51.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.824+0000 7f6d3ba71640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:51.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.824+0000 7f6d3ba71640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d340b1670 con 0x7f6d340b00f0 2024-03-13T07:34:51.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.824+0000 7f6d3ba71640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:51.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.824+0000 7f6d3ba71640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d340b05a0 con 0x7f6d34140400 2024-03-13T07:34:51.827 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.824+0000 7f6d3ba71640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:51.827 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.824+0000 7f6d3ba71640 10 monclient(hunting): authenticate will time out at 1319.233371s 2024-03-13T07:34:51.848 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.847+0000 7f6d3980d640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f6d340b1e70 0x7f6d340b2250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:35950/0 (socket says 172.21.15.132:35950) 2024-03-13T07:34:51.848 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.847+0000 7f6d3980d640 1 -- 172.21.15.132:0/2506423879 learned_addr learned my addr 172.21.15.132:0/2506423879 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:34:51.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.850+0000 7f6d3880b640 1 -- 172.21.15.132:0/2506423879 <== mon.0 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537248740 0 0) 0x7f6d340b1390 con 0x7f6d340b1e70 2024-03-13T07:34:51.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.850+0000 7f6d3880b640 10 monclient(hunting): _negotiate 2024-03-13T07:34:51.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.850+0000 7f6d3880b640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:51.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.850+0000 7f6d3880b640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:51.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.850+0000 7f6d3880b640 10 monclient(hunting): my global_id is 4257 2024-03-13T07:34:51.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.850+0000 7f6d3880b640 1 -- 172.21.15.132:0/2506423879 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d20003520 con 0x7f6d340b1e70 2024-03-13T07:34:51.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.855+0000 7f6d3880b640 1 -- 172.21.15.132:0/2506423879 <== mon.0 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2250710880 0 0) 0x7f6d20003520 con 0x7f6d340b1e70 2024-03-13T07:34:51.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 1 -- 172.21.15.132:0/2506423879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d34140400 msgr2=0x7f6d34142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:51.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 1 --2- 172.21.15.132:0/2506423879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d34140400 0x7f6d34142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:51.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 1 -- 172.21.15.132:0/2506423879 >> v2:172.21.15.195:3300/0 conn(0x7f6d340b00f0 msgr2=0x7f6d3413fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:51.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 1 --2- 172.21.15.132:0/2506423879 >> v2:172.21.15.195:3300/0 conn(0x7f6d340b00f0 0x7f6d3413fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:51.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient: _finish_hunting 0 2024-03-13T07:34:51.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 1 monclient: found mon.noname-a-legacy 2024-03-13T07:34:51.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:51.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.132:6789/0 2024-03-13T07:34:51.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 1 -- 172.21.15.132:0/2506423879 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d34143010 con 0x7f6d340b1e70 2024-03-13T07:34:51.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient: _finish_auth 0 2024-03-13T07:34:51.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient: _check_auth_tickets 2024-03-13T07:34:51.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:51.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 1 -- 172.21.15.132:0/2506423879 <== mon.0 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1639563336 0 0) 0x7f6d30001670 con 0x7f6d340b1e70 2024-03-13T07:34:51.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:51.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-03-13T07:34:51.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient: dump: 2024-03-13T07:34:51.858 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:51.858 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:51.859 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:51.859 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:51.859 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:51.859 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:51.859 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:51.859 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:51.859 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:51.859 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:51.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] but i'm connected to v1:172.21.15.132:6789/0, reconnecting 2024-03-13T07:34:51.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:51.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 1 -- 172.21.15.132:0/2506423879 >> v1:172.21.15.132:6789/0 conn(0x7f6d340b1e70 legacy=0x7f6d340b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:51.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient: _add_conns ranks=[0,1,2] 2024-03-13T07:34:51.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 1 --2- 172.21.15.132:0/2506423879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d200039b0 0x7f6d20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:51.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient(hunting): picked mon.a con 0x7f6d200039b0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:51.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 1 --2- 172.21.15.132:0/2506423879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d20004800 0x7f6d20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:51.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient(hunting): picked mon.b con 0x7f6d20004800 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:51.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 1 --2- 172.21.15.132:0/2506423879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d34140400 0x7f6d2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:51.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient(hunting): picked mon.c con 0x7f6d34140400 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:51.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:51.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 1 -- 172.21.15.132:0/2506423879 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d340b1670 con 0x7f6d200039b0 2024-03-13T07:34:51.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:51.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 1 -- 172.21.15.132:0/2506423879 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d340b05a0 con 0x7f6d20004800 2024-03-13T07:34:51.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:51.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 1 -- 172.21.15.132:0/2506423879 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d200055b0 con 0x7f6d34140400 2024-03-13T07:34:51.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:51.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.856+0000 7f6d3880b640 10 monclient(hunting): _finish_auth 0 2024-03-13T07:34:51.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.860+0000 7f6d3900c640 1 --2- 172.21.15.132:0/2506423879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d20004800 0x7f6d20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:51.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.860+0000 7f6d3900c640 10 monclient(hunting): get_auth_request con 0x7f6d20004800 auth_method 0 2024-03-13T07:34:51.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.860+0000 7f6d3900c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:51.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.860+0000 7f6d3900c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:51.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.860+0000 7f6d3900c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-03-13T07:34:51.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.860+0000 7f6d3980d640 1 --2- 172.21.15.132:0/2506423879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d200039b0 0x7f6d20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:51.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.861+0000 7f6d3900c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:51.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.861+0000 7f6d3900c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:51.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.861+0000 7f6d3900c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-03-13T07:34:51.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.862+0000 7f6d3a00e640 1 --2- 172.21.15.132:0/2506423879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d34140400 0x7f6d2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:51.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.862+0000 7f6d3980d640 10 monclient(hunting): get_auth_request con 0x7f6d200039b0 auth_method 0 2024-03-13T07:34:51.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.862+0000 7f6d3980d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:51.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.862+0000 7f6d3980d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:51.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.862+0000 7f6d3980d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-03-13T07:34:51.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.863+0000 7f6d3900c640 10 monclient(hunting): handle_auth_done global_id 4257 payload 293 2024-03-13T07:34:51.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.863+0000 7f6d3900c640 1 -- 172.21.15.132:0/2506423879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d34140400 msgr2=0x7f6d2000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:51.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.863+0000 7f6d3900c640 1 --2- 172.21.15.132:0/2506423879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d34140400 0x7f6d2000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:51.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.863+0000 7f6d3900c640 1 -- 172.21.15.132:0/2506423879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d200039b0 msgr2=0x7f6d20003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:51.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.863+0000 7f6d3900c640 1 --2- 172.21.15.132:0/2506423879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d200039b0 0x7f6d20003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:51.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.863+0000 7f6d3900c640 10 monclient: _finish_hunting 0 2024-03-13T07:34:51.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.863+0000 7f6d3900c640 1 monclient: found mon.b 2024-03-13T07:34:51.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.863+0000 7f6d3900c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:51.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.863+0000 7f6d3900c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:51.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.863+0000 7f6d3900c640 1 -- 172.21.15.132:0/2506423879 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d20003520 con 0x7f6d20004800 2024-03-13T07:34:51.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.863+0000 7f6d3900c640 10 monclient: _finish_auth 0 2024-03-13T07:34:51.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.863+0000 7f6d3900c640 10 monclient: _check_auth_tickets 2024-03-13T07:34:51.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.863+0000 7f6d3900c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:51.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.863+0000 7f6d3ba71640 5 monclient: authenticate success, global_id 4257 2024-03-13T07:34:51.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.863+0000 7f6d3ba71640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:51.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.865+0000 7f6d3900c640 1 --2- 172.21.15.132:0/2506423879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d20004800 0x7f6d20004c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6d24009860 tx=0x7f6d20005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20303497f8d95e9b server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:51.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.871+0000 7f6d3880b640 1 -- 172.21.15.132:0/2506423879 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24037070 con 0x7f6d20004800 2024-03-13T07:34:51.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.871+0000 7f6d3880b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:51.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.871+0000 7f6d3880b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:51.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.871+0000 7f6d3880b640 10 monclient: dump: 2024-03-13T07:34:51.872 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:51.872 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:51.872 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:51.873 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:51.873 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:51.873 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:51.873 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:51.873 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:51.873 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:51.873 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:51.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.871+0000 7f6d3880b640 1 -- 172.21.15.132:0/2506423879 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d24032620 con 0x7f6d20004800 2024-03-13T07:34:51.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.871+0000 7f6d3ba71640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:51.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.871+0000 7f6d3880b640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:51.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.871+0000 7f6d3ba71640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:51.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.871+0000 7f6d3880b640 1 -- 172.21.15.132:0/2506423879 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24009460 con 0x7f6d20004800 2024-03-13T07:34:51.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.871+0000 7f6d3ba71640 4 set_mon_vals no callback set 2024-03-13T07:34:51.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.871+0000 7f6d3880b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:51.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.871+0000 7f6d3880b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:51.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.871+0000 7f6d3880b640 10 monclient: dump: 2024-03-13T07:34:51.874 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:51.874 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:51.874 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:51.874 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:51.874 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:51.874 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:51.874 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:51.874 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:51.874 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:51.874 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:51.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.872+0000 7f6d3ba71640 10 monclient: shutdown 2024-03-13T07:34:51.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.872+0000 7f6d3ba71640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:51.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.872+0000 7f6d3ba71640 1 -- 172.21.15.132:0/2506423879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d20004800 msgr2=0x7f6d20004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:51.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.872+0000 7f6d3ba71640 1 --2- 172.21.15.132:0/2506423879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d20004800 0x7f6d20004c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6d24009860 tx=0x7f6d20005830 comp rx=0 tx=0).stop 2024-03-13T07:34:51.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.872+0000 7f6d3a00e640 1 -- 172.21.15.132:0/2506423879 reap_dead start 2024-03-13T07:34:51.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.872+0000 7f6d3ba71640 1 -- 172.21.15.132:0/2506423879 shutdown_connections 2024-03-13T07:34:51.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.872+0000 7f6d3ba71640 1 -- 172.21.15.132:0/2506423879 >> 172.21.15.132:0/2506423879 conn(0x7f6d340b80e0 msgr2=0x7f6d340b84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:51.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.872+0000 7f6d3ba71640 1 -- 172.21.15.132:0/2506423879 shutdown_connections 2024-03-13T07:34:51.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.872+0000 7f6d3ba71640 1 -- 172.21.15.132:0/2506423879 wait complete. 2024-03-13T07:34:51.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.873+0000 7f6d3ba71640 10 monclient: build_initial_monmap 2024-03-13T07:34:51.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.873+0000 7f6d3ba71640 10 monclient: monmap: 2024-03-13T07:34:51.878 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:51.878 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:51.878 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:51.878 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:51.879 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:51.879 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:51.879 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:51.879 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:51.879 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:51.879 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:51.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.873+0000 7f6d3ba71640 1 librados: starting msgr at 2024-03-13T07:34:51.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.873+0000 7f6d3ba71640 1 librados: starting objecter 2024-03-13T07:34:51.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.873+0000 7f6d3ba71640 1 Processor -- start 2024-03-13T07:34:51.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.873+0000 7f6d3ba71640 1 -- start start 2024-03-13T07:34:51.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.873+0000 7f6d3ba71640 1 librados: setting wanted keys 2024-03-13T07:34:51.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.873+0000 7f6d3ba71640 1 librados: calling monclient init 2024-03-13T07:34:51.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.873+0000 7f6d3ba71640 10 monclient: init 2024-03-13T07:34:51.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3ba71640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:51.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3ba71640 10 monclient: _add_conns ranks=[0,1,2] 2024-03-13T07:34:51.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3ba71640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d20004800 0x7f6d3413d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:51.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3ba71640 10 monclient(hunting): picked mon.noname-a con 0x7f6d20004800 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:51.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3ba71640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d340ad930 0x7f6d341d7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:51.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3ba71640 10 monclient(hunting): picked mon.noname-b con 0x7f6d340ad930 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:51.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3ba71640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d341d8a20 0x7f6d3411fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:51.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3ba71640 10 monclient(hunting): picked mon.noname-c con 0x7f6d341d8a20 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:51.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3ba71640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:51.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3ba71640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d34143240 con 0x7f6d20004800 2024-03-13T07:34:51.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3ba71640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:51.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3ba71640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d34142d60 con 0x7f6d340ad930 2024-03-13T07:34:51.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3ba71640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:51.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3ba71640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d341433c0 con 0x7f6d341d8a20 2024-03-13T07:34:51.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3ba71640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:51.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3ba71640 10 monclient(hunting): authenticate will time out at 1319.283498s 2024-03-13T07:34:51.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.874+0000 7f6d3980d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d20004800 0x7f6d3413d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:51.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.875+0000 7f6d3980d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d20004800 0x7f6d3413d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56498/0 (socket says 172.21.15.132:56498) 2024-03-13T07:34:51.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.875+0000 7f6d3980d640 1 -- 172.21.15.132:0/2299193643 learned_addr learned my addr 172.21.15.132:0/2299193643 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:51.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.875+0000 7f6d3980d640 10 monclient(hunting): get_auth_request con 0x7f6d20004800 auth_method 0 2024-03-13T07:34:51.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.875+0000 7f6d3980d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:51.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.875+0000 7f6d3980d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:51.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.875+0000 7f6d3980d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:51.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.875+0000 7f6d3980d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:51.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.875+0000 7f6d3980d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:51.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.875+0000 7f6d3980d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:51.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.876+0000 7f6d3a00e640 1 --2- 172.21.15.132:0/2299193643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d341d8a20 0x7f6d3411fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:51.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.876+0000 7f6d3a00e640 10 monclient(hunting): get_auth_request con 0x7f6d341d8a20 auth_method 0 2024-03-13T07:34:51.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.876+0000 7f6d3a00e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:51.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.876+0000 7f6d3a00e640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:51.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.876+0000 7f6d3a00e640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:51.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.877+0000 7f6d3900c640 1 --2- 172.21.15.132:0/2299193643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d340ad930 0x7f6d341d7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:51.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.877+0000 7f6d3a00e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:51.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.877+0000 7f6d3a00e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:51.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.877+0000 7f6d3a00e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:51.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.877+0000 7f6d3900c640 10 monclient(hunting): get_auth_request con 0x7f6d340ad930 auth_method 0 2024-03-13T07:34:51.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.877+0000 7f6d3900c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:51.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.877+0000 7f6d3900c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:51.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.877+0000 7f6d3900c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:51.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.878+0000 7f6d3900c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:51.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.878+0000 7f6d3900c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:51.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.878+0000 7f6d3900c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:51.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.878+0000 7f6d3980d640 10 monclient(hunting): handle_auth_done global_id 4260 payload 824 2024-03-13T07:34:51.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.878+0000 7f6d3980d640 1 -- 172.21.15.132:0/2299193643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d341d8a20 msgr2=0x7f6d3411fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:51.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.878+0000 7f6d3980d640 1 --2- 172.21.15.132:0/2299193643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d341d8a20 0x7f6d3411fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:51.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.878+0000 7f6d3980d640 1 -- 172.21.15.132:0/2299193643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d340ad930 msgr2=0x7f6d341d7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:51.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.878+0000 7f6d3980d640 1 --2- 172.21.15.132:0/2299193643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d340ad930 0x7f6d341d7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:51.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.878+0000 7f6d3980d640 10 monclient: _finish_hunting 0 2024-03-13T07:34:51.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.878+0000 7f6d3980d640 1 monclient: found mon.noname-a 2024-03-13T07:34:51.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.878+0000 7f6d3980d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:51.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.878+0000 7f6d3980d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:51.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.878+0000 7f6d3980d640 1 -- 172.21.15.132:0/2299193643 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d34120200 con 0x7f6d20004800 2024-03-13T07:34:51.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.881+0000 7f6d3980d640 1 --2- 172.21.15.132:0/2299193643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d20004800 0x7f6d3413d3e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000e8a0 tx=0x7f6d3000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:51.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.881+0000 7f6d2a7fc640 10 client.?.objecter ms_handle_connect 0x7f6d20004800 2024-03-13T07:34:51.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.881+0000 7f6d2a7fc640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:51.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.884+0000 7f6d2a7fc640 1 -- 172.21.15.132:0/2299193643 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3000c870 con 0x7f6d20004800 2024-03-13T07:34:51.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.884+0000 7f6d2a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:51.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.884+0000 7f6d2a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:51.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.884+0000 7f6d2a7fc640 10 monclient: dump: 2024-03-13T07:34:51.886 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:51.886 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:51.886 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:51.886 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:51.886 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:51.886 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:51.886 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:51.886 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:51.886 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:51.886 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:51.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.884+0000 7f6d2a7fc640 10 monclient: _finish_auth 0 2024-03-13T07:34:51.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.884+0000 7f6d2a7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:34:51.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.884+0000 7f6d2a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:51.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.884+0000 7f6d2a7fc640 1 -- 172.21.15.132:0/2299193643 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d30018070 con 0x7f6d20004800 2024-03-13T07:34:51.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.884+0000 7f6d2a7fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:51.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.884+0000 7f6d2a7fc640 1 -- 172.21.15.132:0/2299193643 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3000cc70 con 0x7f6d20004800 2024-03-13T07:34:51.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.884+0000 7f6d2a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:51.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.885+0000 7f6d2a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:51.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.885+0000 7f6d2a7fc640 10 monclient: dump: 2024-03-13T07:34:51.887 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:51.887 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:51.887 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:51.887 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:51.887 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:51.887 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:51.887 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:51.888 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:51.888 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:51.888 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:51.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.885+0000 7f6d3ba71640 5 monclient: authenticate success, global_id 4260 2024-03-13T07:34:51.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.885+0000 7f6d3ba71640 10 monclient: _renew_subs 2024-03-13T07:34:51.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.885+0000 7f6d3ba71640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:51.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.885+0000 7f6d3ba71640 1 -- 172.21.15.132:0/2299193643 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d341db3e0 con 0x7f6d20004800 2024-03-13T07:34:51.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.884+0000 7f6d2bfff640 4 set_mon_vals no callback set 2024-03-13T07:34:51.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.885+0000 7f6d3ba71640 10 client.4260.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:51.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.885+0000 7f6d3ba71640 10 monclient: _renew_subs 2024-03-13T07:34:51.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.885+0000 7f6d3ba71640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:51.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.885+0000 7f6d3ba71640 1 -- 172.21.15.132:0/2299193643 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d341db7f0 con 0x7f6d20004800 2024-03-13T07:34:51.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.885+0000 7f6d3ba71640 1 librados: init done 2024-03-13T07:34:51.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.886+0000 7f6d3ba71640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:51.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.886+0000 7f6d3ba71640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:51.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.886+0000 7f6d3ba71640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:51.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.886+0000 7f6d3ba71640 1 -- 172.21.15.132:0/2299193643 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d341d8e00 con 0x7f6d20004800 2024-03-13T07:34:51.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.905+0000 7f6d2a7fc640 1 -- 172.21.15.132:0/2299193643 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f6d30026430 con 0x7f6d20004800 2024-03-13T07:34:51.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.906+0000 7f6d2a7fc640 1 --2- 172.21.15.132:0/2299193643 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f6d1806d590 0x7f6d1806fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:51.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.906+0000 7f6d3900c640 1 --2- 172.21.15.132:0/2299193643 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f6d1806d590 0x7f6d1806fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:51.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.906+0000 7f6d3900c640 10 monclient: get_auth_request con 0x7f6d1806d590 auth_method 0 2024-03-13T07:34:51.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.907+0000 7f6d3900c640 1 --2- 172.21.15.132:0/2299193643 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f6d1806d590 0x7f6d1806fa50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6d24034000 tx=0x7f6d24008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:51.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:51.907+0000 7f6d2a7fc640 10 client.4260.objecter ms_handle_connect 0x7f6d1806d590 2024-03-13T07:34:52.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.179+0000 7f6d2a7fc640 1 -- 172.21.15.132:0/2299193643 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6d3000ce10 con 0x7f6d20004800 2024-03-13T07:34:52.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.179+0000 7f6d2a7fc640 10 client.4260.objecter ms_dispatch 0x7f6d34000b90 osd_map(24..24 src has 1..24) v4 2024-03-13T07:34:52.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.179+0000 7f6d2a7fc640 3 client.4260.objecter handle_osd_map got epochs [24,24] > 0 2024-03-13T07:34:52.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.179+0000 7f6d2a7fc640 3 client.4260.objecter handle_osd_map decoding full epoch 24 2024-03-13T07:34:52.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.179+0000 7f6d2a7fc640 20 client.4260.objecter dump_active .. 0 homeless 2024-03-13T07:34:52.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.179+0000 7f6d2a7fc640 1 -- 172.21.15.132:0/2299193643 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f6d30059550 con 0x7f6d20004800 2024-03-13T07:34:52.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.179+0000 7f6d2a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:52.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.179+0000 7f6d2a7fc640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:52.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.424+0000 7f6d3ba71640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2024-03-13T07:34:52.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.424+0000 7f6d3ba71640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2024-03-13T07:34:52.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.424+0000 7f6d3ba71640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:52.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.424+0000 7f6d3ba71640 1 -- 172.21.15.132:0/2299193643 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6d341203d0 con 0x7f6d20004800 2024-03-13T07:34:52.566 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.566+0000 7f6d2a7fc640 1 -- 172.21.15.132:0/2299193643 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+172585 (secure 0 0 0) 0x7f6d3005d5f0 con 0x7f6d20004800 2024-03-13T07:34:52.567 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.566+0000 7f6d2a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2024-03-13T07:34:52.567 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.566+0000 7f6d2a7fc640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:52.569 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:34:52.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 10 librados: watch_flush enter 2024-03-13T07:34:52.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 10 librados: watch_flush exit 2024-03-13T07:34:52.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 20 client.4260.objecter shutdown clearing up homeless session... 2024-03-13T07:34:52.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 10 client.4260.objecter successfully canceled tick 2024-03-13T07:34:52.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 1 -- 172.21.15.132:0/2299193643 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f6d1806d590 msgr2=0x7f6d1806fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:52.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 1 --2- 172.21.15.132:0/2299193643 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f6d1806d590 0x7f6d1806fa50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6d24034000 tx=0x7f6d24008030 comp rx=0 tx=0).stop 2024-03-13T07:34:52.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 10 monclient: shutdown 2024-03-13T07:34:52.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:52.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 1 -- 172.21.15.132:0/2299193643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d20004800 msgr2=0x7f6d3413d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:52.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 1 --2- 172.21.15.132:0/2299193643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d20004800 0x7f6d3413d3e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000e8a0 tx=0x7f6d3000ed60 comp rx=0 tx=0).stop 2024-03-13T07:34:52.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 1 -- 172.21.15.132:0/2299193643 shutdown_connections 2024-03-13T07:34:52.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 1 --2- 172.21.15.132:0/2299193643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d341d8a20 0x7f6d3411fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:52.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 1 --2- 172.21.15.132:0/2299193643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d340ad930 0x7f6d341d7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:52.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 1 --2- 172.21.15.132:0/2299193643 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f6d1806d590 0x7f6d1806fa50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:52.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 1 --2- 172.21.15.132:0/2299193643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d20004800 0x7f6d3413d3e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:52.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 1 -- 172.21.15.132:0/2299193643 >> 172.21.15.132:0/2299193643 conn(0x7f6d340b80e0 msgr2=0x7f6d340b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:52.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 1 -- 172.21.15.132:0/2299193643 shutdown_connections 2024-03-13T07:34:52.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 1 -- 172.21.15.132:0/2299193643 wait complete. 2024-03-13T07:34:52.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.569+0000 7f6d3ba71640 1 librados: shutdown 2024-03-13T07:34:52.580 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":3847550149},{"type":"v1","addr":"172.21.15.132:6801","nonce":3847550149}]},"active_addr":"172.21.15.132:6801/3847550149","active_change":"2024-03-13T07:33:52.862565+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":2473307426}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":3111378430}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":230174643}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":3113513601}]}]} 2024-03-13T07:34:52.587 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-13T07:34:52.587 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-13T07:34:52.587 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-13T07:34:52.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.688+0000 7fb74ae85640 10 monclient: get_monmap_and_config 2024-03-13T07:34:52.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.688+0000 7fb74ae85640 10 monclient: build_initial_monmap 2024-03-13T07:34:52.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.688+0000 7fb74ae85640 10 monclient: monmap: 2024-03-13T07:34:52.689 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:52.689 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:52.689 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:52.690355+0000 2024-03-13T07:34:52.690 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:52.690355+0000 2024-03-13T07:34:52.690 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:52.690 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:52.690 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:52.690 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:52.690 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:52.690 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:52.690 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:52.690 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:52.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.689+0000 7fb74ae85640 1 Processor -- start 2024-03-13T07:34:52.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.689+0000 7fb74ae85640 1 -- start start 2024-03-13T07:34:52.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.689+0000 7fb74ae85640 10 monclient: init 2024-03-13T07:34:52.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.689+0000 7fb74ae85640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:52.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.689+0000 7fb74ae85640 10 monclient: _add_conns ranks=[1,0,4,3,2] 2024-03-13T07:34:52.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.689+0000 7fb74ae85640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb744070ba0 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:52.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.689+0000 7fb74ae85640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb744071d90 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:52.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.689+0000 7fb74ae85640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb74413dc90 0x7fb744140cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:52.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.689+0000 7fb74ae85640 10 monclient(hunting): picked mon.noname-c con 0x7fb74413dc90 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:52.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.689+0000 7fb74ae85640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb744072df0 con 0x7fb744071d90 2024-03-13T07:34:52.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.689+0000 7fb74ae85640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7440b02a0 con 0x7fb744070ba0 2024-03-13T07:34:52.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.689+0000 7fb74ae85640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:52.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.689+0000 7fb74ae85640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb744070f80 con 0x7fb74413dc90 2024-03-13T07:34:52.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.689+0000 7fb74ae85640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:52.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.689+0000 7fb74ae85640 10 monclient(hunting): authenticate will time out at 1320.098997s 2024-03-13T07:34:52.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.690+0000 7fb743fff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fb744071d90 0x7fb7441405e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:35954/0 (socket says 172.21.15.132:35954) 2024-03-13T07:34:52.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.690+0000 7fb743fff640 1 -- 172.21.15.132:0/3870449873 learned_addr learned my addr 172.21.15.132:0/3870449873 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:34:52.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.690+0000 7fb749422640 1 --2- 172.21.15.132:0/3870449873 >> v2:172.21.15.195:3300/0 conn(0x7fb74413dc90 0x7fb744140cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:52.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.690+0000 7fb749422640 10 monclient(hunting): get_auth_request con 0x7fb74413dc90 auth_method 0 2024-03-13T07:34:52.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.690+0000 7fb749422640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:52.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.690+0000 7fb749422640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:52.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.690+0000 7fb749422640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:52.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.691+0000 7fb749422640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:52.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.691+0000 7fb749422640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:52.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.691+0000 7fb749422640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:52.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.692+0000 7fb7437fe640 1 -- 172.21.15.132:0/3870449873 <== mon.0 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 524068417 0 0) 0x7fb744072df0 con 0x7fb744071d90 2024-03-13T07:34:52.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.693+0000 7fb7437fe640 10 monclient(hunting): _negotiate 2024-03-13T07:34:52.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.693+0000 7fb7437fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:52.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.693+0000 7fb7437fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:52.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.693+0000 7fb7437fe640 10 monclient(hunting): my global_id is 4263 2024-03-13T07:34:52.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.693+0000 7fb7437fe640 1 -- 172.21.15.132:0/3870449873 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb734003520 con 0x7fb744071d90 2024-03-13T07:34:52.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.693+0000 7fb749422640 10 monclient(hunting): handle_auth_done global_id 4267 payload 274 2024-03-13T07:34:52.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.693+0000 7fb749422640 1 -- 172.21.15.132:0/3870449873 >> v1:172.21.15.195:6789/0 conn(0x7fb744070ba0 legacy=0x7fb74413d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:52.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.693+0000 7fb749422640 1 -- 172.21.15.132:0/3870449873 >> v1:172.21.15.132:6789/0 conn(0x7fb744071d90 legacy=0x7fb7441405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:52.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.693+0000 7fb749422640 10 monclient: _finish_hunting 0 2024-03-13T07:34:52.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.693+0000 7fb749422640 1 monclient: found mon.noname-c 2024-03-13T07:34:52.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.693+0000 7fb749422640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:52.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.693+0000 7fb749422640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T07:34:52.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.693+0000 7fb749422640 1 -- 172.21.15.132:0/3870449873 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb744141230 con 0x7fb74413dc90 2024-03-13T07:34:52.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.697+0000 7fb749422640 1 --2- 172.21.15.132:0/3870449873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb74413dc90 0x7fb744140cf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009d90 tx=0x7fb73c02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6d3da0161cee3b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:52.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb7437fe640 1 -- 172.21.15.132:0/3870449873 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c039070 con 0x7fb74413dc90 2024-03-13T07:34:52.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb7437fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:52.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb7437fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:52.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb7437fe640 10 monclient: dump: 2024-03-13T07:34:52.702 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:52.702 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:52.702 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:52.702 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:52.702 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:52.703 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:52.703 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:52.703 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:52.703 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:52.703 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:52.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb7437fe640 10 monclient: _finish_auth 0 2024-03-13T07:34:52.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb7437fe640 10 monclient: _check_auth_tickets 2024-03-13T07:34:52.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb7437fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:52.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb7437fe640 1 -- 172.21.15.132:0/3870449873 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73c02cc60 con 0x7fb74413dc90 2024-03-13T07:34:52.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb74ae85640 5 monclient: authenticate success, global_id 4267 2024-03-13T07:34:52.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb74ae85640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:52.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb7437fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:52.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb74ae85640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:52.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb7437fe640 1 -- 172.21.15.132:0/3870449873 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c034660 con 0x7fb74413dc90 2024-03-13T07:34:52.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb7437fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:52.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb74ae85640 4 set_mon_vals no callback set 2024-03-13T07:34:52.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb7437fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:52.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb7437fe640 10 monclient: dump: 2024-03-13T07:34:52.704 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:52.704 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:52.704 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:52.704 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:52.704 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:52.704 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:52.704 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:52.704 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:52.704 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:52.704 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:52.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb74ae85640 10 monclient: shutdown 2024-03-13T07:34:52.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb74ae85640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:52.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb74ae85640 1 -- 172.21.15.132:0/3870449873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb74413dc90 msgr2=0x7fb744140cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:52.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb74ae85640 1 --2- 172.21.15.132:0/3870449873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb74413dc90 0x7fb744140cf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009d90 tx=0x7fb73c02beb0 comp rx=0 tx=0).stop 2024-03-13T07:34:52.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb74ae85640 1 -- 172.21.15.132:0/3870449873 shutdown_connections 2024-03-13T07:34:52.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb74ae85640 1 --2- 172.21.15.132:0/3870449873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb74413dc90 0x7fb744140cf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:52.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb74ae85640 1 -- 172.21.15.132:0/3870449873 >> 172.21.15.132:0/3870449873 conn(0x7fb7440b5d70 msgr2=0x7fb7440b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:52.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb74ae85640 1 -- 172.21.15.132:0/3870449873 shutdown_connections 2024-03-13T07:34:52.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.702+0000 7fb74ae85640 1 -- 172.21.15.132:0/3870449873 wait complete. 2024-03-13T07:34:52.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.703+0000 7fb74ae85640 10 monclient: build_initial_monmap 2024-03-13T07:34:52.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.703+0000 7fb74ae85640 10 monclient: monmap: 2024-03-13T07:34:52.708 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:52.708 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:52.708 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:52.708 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:52.708 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:52.709 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:52.709 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:52.709 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:52.709 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:52.709 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:52.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.703+0000 7fb74ae85640 1 librados: starting msgr at 2024-03-13T07:34:52.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.703+0000 7fb74ae85640 1 librados: starting objecter 2024-03-13T07:34:52.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.703+0000 7fb74ae85640 1 Processor -- start 2024-03-13T07:34:52.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.703+0000 7fb74ae85640 1 -- start start 2024-03-13T07:34:52.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.703+0000 7fb74ae85640 1 librados: setting wanted keys 2024-03-13T07:34:52.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.703+0000 7fb74ae85640 1 librados: calling monclient init 2024-03-13T07:34:52.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.703+0000 7fb74ae85640 10 monclient: init 2024-03-13T07:34:52.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb74ae85640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:52.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb74ae85640 10 monclient: _add_conns ranks=[2,0,1] 2024-03-13T07:34:52.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb74ae85640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb744070ba0 0x7fb7440b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:52.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb74ae85640 10 monclient(hunting): picked mon.noname-c con 0x7fb744070ba0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:52.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb74ae85640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb744071d90 0x7fb74406bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:52.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb74ae85640 10 monclient(hunting): picked mon.noname-a con 0x7fb744071d90 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:52.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb74ae85640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb74413dc90 0x7fb74412ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:52.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb74ae85640 10 monclient(hunting): picked mon.noname-b con 0x7fb74413dc90 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:52.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb74ae85640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:52.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb74ae85640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74414bd50 con 0x7fb744071d90 2024-03-13T07:34:52.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb74ae85640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:52.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb74ae85640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb744071340 con 0x7fb74413dc90 2024-03-13T07:34:52.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb74ae85640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:52.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb74ae85640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74414bed0 con 0x7fb744070ba0 2024-03-13T07:34:52.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb74ae85640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:52.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb74ae85640 10 monclient(hunting): authenticate will time out at 1320.113286s 2024-03-13T07:34:52.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb743fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb744071d90 0x7fb74406bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:52.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.704+0000 7fb748c21640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb744070ba0 0x7fb7440b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:52.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb743fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb744071d90 0x7fb74406bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56512/0 (socket says 172.21.15.132:56512) 2024-03-13T07:34:52.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb743fff640 1 -- 172.21.15.132:0/1609358454 learned_addr learned my addr 172.21.15.132:0/1609358454 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:52.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb743fff640 10 monclient(hunting): get_auth_request con 0x7fb744071d90 auth_method 0 2024-03-13T07:34:52.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb743fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:52.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb743fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:52.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb743fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:52.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb748c21640 10 monclient(hunting): get_auth_request con 0x7fb744070ba0 auth_method 0 2024-03-13T07:34:52.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb748c21640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:52.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb748c21640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:52.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb748c21640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:52.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb749422640 1 --2- 172.21.15.132:0/1609358454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb74413dc90 0x7fb74412ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:52.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb749422640 10 monclient(hunting): get_auth_request con 0x7fb74413dc90 auth_method 0 2024-03-13T07:34:52.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb749422640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:52.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb749422640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:52.712 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb749422640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:52.712 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb743fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:52.712 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb743fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:52.712 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb743fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:52.712 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb748c21640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:52.712 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb748c21640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:52.712 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.705+0000 7fb748c21640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:52.712 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.706+0000 7fb749422640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:52.712 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.706+0000 7fb749422640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:52.712 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.706+0000 7fb749422640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:52.713 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.708+0000 7fb749422640 10 monclient(hunting): handle_auth_done global_id 4273 payload 824 2024-03-13T07:34:52.713 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.709+0000 7fb749422640 1 -- 172.21.15.132:0/1609358454 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb744070ba0 msgr2=0x7fb7440b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:52.713 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.709+0000 7fb749422640 1 --2- 172.21.15.132:0/1609358454 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb744070ba0 0x7fb7440b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:52.713 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.709+0000 7fb749422640 1 -- 172.21.15.132:0/1609358454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb744071d90 msgr2=0x7fb74406bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:52.713 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.709+0000 7fb749422640 1 --2- 172.21.15.132:0/1609358454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb744071d90 0x7fb74406bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:52.713 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.709+0000 7fb749422640 10 monclient: _finish_hunting 0 2024-03-13T07:34:52.713 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.709+0000 7fb749422640 1 monclient: found mon.noname-b 2024-03-13T07:34:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.709+0000 7fb749422640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.709+0000 7fb749422640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.709+0000 7fb749422640 1 -- 172.21.15.132:0/1609358454 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74406c610 con 0x7fb74413dc90 2024-03-13T07:34:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.711+0000 7fb749422640 1 --2- 172.21.15.132:0/1609358454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb74413dc90 0x7fb74412ef40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb73c033000 tx=0x7fb73c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.711+0000 7fb7417fa640 10 client.?.objecter ms_handle_connect 0x7fb74413dc90 2024-03-13T07:34:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.711+0000 7fb7417fa640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:52.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb7417fa640 1 -- 172.21.15.132:0/1609358454 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c044020 con 0x7fb74413dc90 2024-03-13T07:34:52.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb7417fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:52.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb7417fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb7417fa640 10 monclient: dump: 2024-03-13T07:34:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:52.716 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb7417fa640 10 monclient: _finish_auth 0 2024-03-13T07:34:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb7417fa640 10 monclient: _check_auth_tickets 2024-03-13T07:34:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb7417fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb7417fa640 1 -- 172.21.15.132:0/1609358454 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73c03f470 con 0x7fb74413dc90 2024-03-13T07:34:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb74ae85640 5 monclient: authenticate success, global_id 4273 2024-03-13T07:34:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb74ae85640 10 monclient: _renew_subs 2024-03-13T07:34:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb74ae85640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb74ae85640 1 -- 172.21.15.132:0/1609358454 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74406c310 con 0x7fb74413dc90 2024-03-13T07:34:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb7417fa640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb7417fa640 1 -- 172.21.15.132:0/1609358454 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c039040 con 0x7fb74413dc90 2024-03-13T07:34:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb7417fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb7417fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb7417fa640 10 monclient: dump: 2024-03-13T07:34:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:52.718 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:52.718 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:52.718 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:52.718 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:52.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb742ffd640 4 set_mon_vals no callback set 2024-03-13T07:34:52.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb74ae85640 10 client.4273.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:52.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb74ae85640 10 monclient: _renew_subs 2024-03-13T07:34:52.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb74ae85640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:52.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.714+0000 7fb74ae85640 1 -- 172.21.15.132:0/1609358454 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74412f480 con 0x7fb74413dc90 2024-03-13T07:34:52.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.715+0000 7fb74ae85640 1 librados: init done 2024-03-13T07:34:52.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.715+0000 7fb74ae85640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:52.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.715+0000 7fb74ae85640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:52.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.715+0000 7fb74ae85640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:52.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.715+0000 7fb74ae85640 1 -- 172.21.15.132:0/1609358454 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74404f000 con 0x7fb74413dc90 2024-03-13T07:34:52.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.735+0000 7fb7417fa640 1 -- 172.21.15.132:0/1609358454 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb73c047020 con 0x7fb74413dc90 2024-03-13T07:34:52.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.735+0000 7fb7417fa640 1 --2- 172.21.15.132:0/1609358454 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb72806d560 0x7fb72806fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:52.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.735+0000 7fb748c21640 1 --2- 172.21.15.132:0/1609358454 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb72806d560 0x7fb72806fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:52.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.735+0000 7fb748c21640 10 monclient: get_auth_request con 0x7fb72806d560 auth_method 0 2024-03-13T07:34:52.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.735+0000 7fb7417fa640 10 client.4273.objecter ms_handle_connect 0x7fb72806d560 2024-03-13T07:34:52.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.735+0000 7fb748c21640 1 --2- 172.21.15.132:0/1609358454 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb72806d560 0x7fb72806fa20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb738001780 tx=0x7fb738009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:52.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.991+0000 7fb7417fa640 1 -- 172.21.15.132:0/1609358454 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb73c079cd0 con 0x7fb74413dc90 2024-03-13T07:34:52.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.991+0000 7fb7417fa640 10 client.4273.objecter ms_dispatch 0x7fb744000b90 osd_map(24..24 src has 1..24) v4 2024-03-13T07:34:52.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.991+0000 7fb7417fa640 3 client.4273.objecter handle_osd_map got epochs [24,24] > 0 2024-03-13T07:34:52.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.991+0000 7fb7417fa640 3 client.4273.objecter handle_osd_map decoding full epoch 24 2024-03-13T07:34:52.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:52.991+0000 7fb7417fa640 20 client.4273.objecter dump_active .. 0 homeless 2024-03-13T07:34:53.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.001+0000 7fb7417fa640 1 -- 172.21.15.132:0/1609358454 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fb73c04e030 con 0x7fb74413dc90 2024-03-13T07:34:53.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.001+0000 7fb7417fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:53.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.001+0000 7fb7417fa640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:53.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.205+0000 7fb74ae85640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:53.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.205+0000 7fb74ae85640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:53.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.205+0000 7fb74ae85640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:53.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.205+0000 7fb74ae85640 1 -- 172.21.15.132:0/1609358454 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb74412ff80 con 0x7fb74413dc90 2024-03-13T07:34:53.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.245+0000 7fb7417fa640 1 -- 172.21.15.132:0/1609358454 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+14766 (secure 0 0 0) 0x7fb73c07bc30 con 0x7fb74413dc90 2024-03-13T07:34:53.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.245+0000 7fb7417fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:53.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.245+0000 7fb7417fa640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:53.246 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:34:53.246 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":25,"fsid":"74e4bc2f-6a5e-4423-a8c8-807970a2f96c","created":"2024-03-13T07:33:47.494293+0000","modified":"2024-03-13T07:34:52.782902+0000","last_up_change":"2024-03-13T07:34:44.296009+0000","last_in_change":"2024-03-13T07:33:53.171597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-13T07:34:33.504752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-13T07:34:49.707222+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0edb216c-bc1e-4865-aafe-218611f3eb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":590522438},{"type":"v1","addr":"172.21.15.132:6819","nonce":590522438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":590522438},{"type":"v1","addr":"172.21.15.132:6821","nonce":590522438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":590522438},{"type":"v1","addr":"172.21.15.132:6825","nonce":590522438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":590522438},{"type":"v1","addr":"172.21.15.132:6823","nonce":590522438}]},"public_addr":"172.21.15.132:6819/590522438","cluster_addr":"172.21.15.132:6821/590522438","heartbeat_back_addr":"172.21.15.132:6825/590522438","heartbeat_front_addr":"172.21.15.132:6823/590522438","state":["exists","up"]},{"osd":1,"uuid":"6be0ae6b-951e-4279-ba0a-9b05b6e6e85e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1071068206},{"type":"v1","addr":"172.21.15.132:6803","nonce":1071068206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1071068206},{"type":"v1","addr":"172.21.15.132:6805","nonce":1071068206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1071068206},{"type":"v1","addr":"172.21.15.132:6809","nonce":1071068206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1071068206},{"type":"v1","addr":"172.21.15.132:6807","nonce":1071068206}]},"public_addr":"172.21.15.132:6803/1071068206","cluster_addr":"172.21.15.132:6805/1071068206","heartbeat_back_addr":"172.21.15.132:6809/1071068206","heartbeat_front_addr":"172.21.15.132:6807/1071068206","state":["exists","up"]},{"osd":2,"uuid":"18008b7f-cfe7-4450-a3cf-aab63ebbe99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":958254560},{"type":"v1","addr":"172.21.15.132:6827","nonce":958254560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":958254560},{"type":"v1","addr":"172.21.15.132:6829","nonce":958254560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":958254560},{"type":"v1","addr":"172.21.15.132:6833","nonce":958254560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":958254560},{"type":"v1","addr":"172.21.15.132:6831","nonce":958254560}]},"public_addr":"172.21.15.132:6827/958254560","cluster_addr":"172.21.15.132:6829/958254560","heartbeat_back_addr":"172.21.15.132:6833/958254560","heartbeat_front_addr":"172.21.15.132:6831/958254560","state":["exists","up"]},{"osd":3,"uuid":"411b6066-4b9f-40e7-b92d-4ba914eab63e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6811","nonce":1770057764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6813","nonce":1770057764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6817","nonce":1770057764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6815","nonce":1770057764}]},"public_addr":"172.21.15.132:6811/1770057764","cluster_addr":"172.21.15.132:6813/1770057764","heartbeat_back_addr":"172.21.15.132:6817/1770057764","heartbeat_front_addr":"172.21.15.132:6815/1770057764","state":["exists","up"]},{"osd":4,"uuid":"1a719246-ae04-42b0-bd9c-fc318dc57ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":699638467},{"type":"v1","addr":"172.21.15.195:6803","nonce":699638467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":699638467},{"type":"v1","addr":"172.21.15.195:6809","nonce":699638467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":699638467},{"type":"v1","addr":"172.21.15.195:6815","nonce":699638467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6811","nonce":699638467},{"type":"v1","addr":"172.21.15.195:6813","nonce":699638467}]},"public_addr":"172.21.15.195:6803/699638467","cluster_addr":"172.21.15.195:6809/699638467","heartbeat_back_addr":"172.21.15.195:6815/699638467","heartbeat_front_addr":"172.21.15.195:6813/699638467","state":["exists","up"]},{"osd":5,"uuid":"9d9ee991-241b-4507-ba2f-e3b5e2844d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6817","nonce":1953907706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6819","nonce":1953907706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6825","nonce":1953907706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6821","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6822","nonce":1953907706}]},"public_addr":"172.21.15.195:6817/1953907706","cluster_addr":"172.21.15.195:6819/1953907706","heartbeat_back_addr":"172.21.15.195:6825/1953907706","heartbeat_front_addr":"172.21.15.195:6822/1953907706","state":["exists","up"]},{"osd":6,"uuid":"af8b34fd-927b-4860-971b-9fddf699a0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6801","nonce":4106887608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6805","nonce":4106887608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6812","nonce":4106887608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6808","nonce":4106887608}]},"public_addr":"172.21.15.195:6801/4106887608","cluster_addr":"172.21.15.195:6805/4106887608","heartbeat_back_addr":"172.21.15.195:6812/4106887608","heartbeat_front_addr":"172.21.15.195:6808/4106887608","state":["exists","up"]},{"osd":7,"uuid":"19a39645-8664-463b-bae3-d7aa1026e5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6823","nonce":3378300314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6827","nonce":3378300314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6831","nonce":3378300314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6829","nonce":3378300314}]},"public_addr":"172.21.15.195:6823/3378300314","cluster_addr":"172.21.15.195:6827/3378300314","heartbeat_back_addr":"172.21.15.195:6831/3378300314","heartbeat_front_addr":"172.21.15.195:6829/3378300314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:32.324299+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:31.786840+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:32.474925+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:31.877950+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:30.195463+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:30.243386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:30.409185+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:29.949919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T07:34:53.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 10 librados: watch_flush enter 2024-03-13T07:34:53.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 10 librados: watch_flush exit 2024-03-13T07:34:53.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 20 client.4273.objecter shutdown clearing up homeless session... 2024-03-13T07:34:53.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 10 client.4273.objecter successfully canceled tick 2024-03-13T07:34:53.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 1 -- 172.21.15.132:0/1609358454 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb72806d560 msgr2=0x7fb72806fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:53.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 1 --2- 172.21.15.132:0/1609358454 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb72806d560 0x7fb72806fa20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb738001780 tx=0x7fb738009040 comp rx=0 tx=0).stop 2024-03-13T07:34:53.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 10 monclient: shutdown 2024-03-13T07:34:53.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:53.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 1 -- 172.21.15.132:0/1609358454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb74413dc90 msgr2=0x7fb74412ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:53.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 1 --2- 172.21.15.132:0/1609358454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb74413dc90 0x7fb74412ef40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb73c033000 tx=0x7fb73c004a40 comp rx=0 tx=0).stop 2024-03-13T07:34:53.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 1 -- 172.21.15.132:0/1609358454 shutdown_connections 2024-03-13T07:34:53.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 1 --2- 172.21.15.132:0/1609358454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb74413dc90 0x7fb74412ef40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:53.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 1 --2- 172.21.15.132:0/1609358454 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb72806d560 0x7fb72806fa20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:53.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 1 --2- 172.21.15.132:0/1609358454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb744071d90 0x7fb74406bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:53.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 1 --2- 172.21.15.132:0/1609358454 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb744070ba0 0x7fb7440b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:53.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 1 -- 172.21.15.132:0/1609358454 >> 172.21.15.132:0/1609358454 conn(0x7fb7440b5d70 msgr2=0x7fb744137190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:53.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 1 -- 172.21.15.132:0/1609358454 shutdown_connections 2024-03-13T07:34:53.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 1 -- 172.21.15.132:0/1609358454 wait complete. 2024-03-13T07:34:53.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.247+0000 7fb74ae85640 1 librados: shutdown 2024-03-13T07:34:53.259 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-13T07:34:53.260 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-13T07:34:53.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.359+0000 7f52481b2640 10 monclient: get_monmap_and_config 2024-03-13T07:34:53.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.359+0000 7f52481b2640 10 monclient: build_initial_monmap 2024-03-13T07:34:53.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.359+0000 7f52481b2640 10 monclient: monmap: 2024-03-13T07:34:53.360 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:53.360 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:53.360 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:53.361067+0000 2024-03-13T07:34:53.360 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:53.361067+0000 2024-03-13T07:34:53.360 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:53.360 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:53.361 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:53.361 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:53.361 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:53.361 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:53.361 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:53.361 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:53.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.359+0000 7f52481b2640 1 Processor -- start 2024-03-13T07:34:53.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.360+0000 7f52481b2640 1 -- start start 2024-03-13T07:34:53.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.360+0000 7f52481b2640 10 monclient: init 2024-03-13T07:34:53.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.360+0000 7f52481b2640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:53.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.360+0000 7f52481b2640 10 monclient: _add_conns ranks=[1,3,0,2,4] 2024-03-13T07:34:53.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.360+0000 7f52481b2640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f52400b1e70 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:53.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.360+0000 7f52481b2640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f52400b2250 0x7f52401427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:53.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.360+0000 7f52481b2640 10 monclient(hunting): picked mon.noname-a con 0x7f52400b2250 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:53.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.360+0000 7f52481b2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5240142d30 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:53.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.360+0000 7f52481b2640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52400b01c0 con 0x7f5240142d30 2024-03-13T07:34:53.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.360+0000 7f52481b2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52400aefe0 con 0x7f52400b1e70 2024-03-13T07:34:53.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.360+0000 7f52481b2640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:53.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.360+0000 7f52481b2640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5240140460 con 0x7f52400b2250 2024-03-13T07:34:53.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.360+0000 7f52481b2640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:53.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.360+0000 7f52481b2640 10 monclient(hunting): authenticate will time out at 1320.769773s 2024-03-13T07:34:53.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.361+0000 7f524574d640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f52400b2250 0x7f52401427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:53.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.361+0000 7f5245f4e640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f52400b1e70 0x7f524013fd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.132:47092/0 (socket says 172.21.15.132:47092) 2024-03-13T07:34:53.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.361+0000 7f5245f4e640 1 -- 172.21.15.132:0/2159852558 learned_addr learned my addr 172.21.15.132:0/2159852558 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:34:53.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.361+0000 7f524574d640 10 monclient(hunting): get_auth_request con 0x7f52400b2250 auth_method 0 2024-03-13T07:34:53.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.361+0000 7f524574d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:53.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.361+0000 7f524574d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:53.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.361+0000 7f524574d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:53.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.362+0000 7f524574d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:53.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.362+0000 7f524574d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:53.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.362+0000 7f524574d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:53.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.363+0000 7f5244f4c640 1 -- 172.21.15.132:0/2159852558 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747255181 0 0) 0x7f52400aefe0 con 0x7f52400b1e70 2024-03-13T07:34:53.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.363+0000 7f5244f4c640 10 monclient(hunting): _negotiate 2024-03-13T07:34:53.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.363+0000 7f5244f4c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:53.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.363+0000 7f5244f4c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:53.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.363+0000 7f5244f4c640 10 monclient(hunting): my global_id is 4276 2024-03-13T07:34:53.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.363+0000 7f5244f4c640 1 -- 172.21.15.132:0/2159852558 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f522c003520 con 0x7f52400b1e70 2024-03-13T07:34:53.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.364+0000 7f524574d640 10 monclient(hunting): handle_auth_done global_id 4269 payload 274 2024-03-13T07:34:53.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.364+0000 7f524574d640 1 -- 172.21.15.132:0/2159852558 >> v1:172.21.15.195:6789/0 conn(0x7f52400b1e70 legacy=0x7f524013fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:53.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.364+0000 7f524574d640 1 -- 172.21.15.132:0/2159852558 >> v1:172.21.15.132:6789/0 conn(0x7f5240142d30 legacy=0x7f524014d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:53.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.364+0000 7f524574d640 10 monclient: _finish_hunting 0 2024-03-13T07:34:53.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.364+0000 7f524574d640 1 monclient: found mon.noname-a 2024-03-13T07:34:53.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.364+0000 7f524574d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:53.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.364+0000 7f524574d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:53.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.364+0000 7f524574d640 1 -- 172.21.15.132:0/2159852558 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52400b4730 con 0x7f52400b2250 2024-03-13T07:34:53.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.366+0000 7f524574d640 1 --2- 172.21.15.132:0/2159852558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52400b2250 0x7f52401427f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f523c0097a0 tx=0x7f523c02b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409c8d60c99a8844 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:53.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.371+0000 7f5244f4c640 1 -- 172.21.15.132:0/2159852558 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c02f810 con 0x7f52400b2250 2024-03-13T07:34:53.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.371+0000 7f5244f4c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:53.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.371+0000 7f5244f4c640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:53.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.371+0000 7f5244f4c640 10 monclient: dump: 2024-03-13T07:34:53.373 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:53.373 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:53.373 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:53.373 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:53.373 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:53.373 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:53.373 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:53.373 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:53.373 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:53.374 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:53.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.371+0000 7f5244f4c640 10 monclient: _finish_auth 0 2024-03-13T07:34:53.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.371+0000 7f5244f4c640 10 monclient: _check_auth_tickets 2024-03-13T07:34:53.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.371+0000 7f5244f4c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:53.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.371+0000 7f5244f4c640 1 -- 172.21.15.132:0/2159852558 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523c02f9b0 con 0x7f52400b2250 2024-03-13T07:34:53.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.371+0000 7f5244f4c640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:53.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.371+0000 7f5244f4c640 1 -- 172.21.15.132:0/2159852558 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c02fd80 con 0x7f52400b2250 2024-03-13T07:34:53.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.371+0000 7f52481b2640 5 monclient: authenticate success, global_id 4269 2024-03-13T07:34:53.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.371+0000 7f52481b2640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:53.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.371+0000 7f52481b2640 4 set_mon_vals no callback set 2024-03-13T07:34:53.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.371+0000 7f5244f4c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:53.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.371+0000 7f5244f4c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:53.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.371+0000 7f5244f4c640 10 monclient: dump: 2024-03-13T07:34:53.375 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:53.375 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:53.375 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:53.375 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:53.375 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:53.375 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:53.375 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:53.375 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:53.375 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:53.375 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:53.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.372+0000 7f52481b2640 10 monclient: shutdown 2024-03-13T07:34:53.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.372+0000 7f52481b2640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:53.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.372+0000 7f52481b2640 1 -- 172.21.15.132:0/2159852558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52400b2250 msgr2=0x7f52401427f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:53.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.372+0000 7f52481b2640 1 --2- 172.21.15.132:0/2159852558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52400b2250 0x7f52401427f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f523c0097a0 tx=0x7f523c02b9f0 comp rx=0 tx=0).stop 2024-03-13T07:34:53.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.372+0000 7f52481b2640 1 -- 172.21.15.132:0/2159852558 shutdown_connections 2024-03-13T07:34:53.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.372+0000 7f52481b2640 1 --2- 172.21.15.132:0/2159852558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52400b2250 0x7f52401427f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:53.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.372+0000 7f52481b2640 1 -- 172.21.15.132:0/2159852558 >> 172.21.15.132:0/2159852558 conn(0x7f52400b7f70 msgr2=0x7f52400b8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:53.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.372+0000 7f52481b2640 1 -- 172.21.15.132:0/2159852558 shutdown_connections 2024-03-13T07:34:53.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.372+0000 7f52481b2640 1 -- 172.21.15.132:0/2159852558 wait complete. 2024-03-13T07:34:53.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.372+0000 7f52481b2640 10 monclient: build_initial_monmap 2024-03-13T07:34:53.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.372+0000 7f52481b2640 10 monclient: monmap: 2024-03-13T07:34:53.379 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:53.379 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:53.379 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:53.379 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:53.379 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:53.379 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:53.379 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:53.379 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:53.379 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:53.380 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:53.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 1 librados: starting msgr at 2024-03-13T07:34:53.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 1 librados: starting objecter 2024-03-13T07:34:53.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 1 Processor -- start 2024-03-13T07:34:53.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 1 -- start start 2024-03-13T07:34:53.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 1 librados: setting wanted keys 2024-03-13T07:34:53.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 1 librados: calling monclient init 2024-03-13T07:34:53.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 10 monclient: init 2024-03-13T07:34:53.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:53.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 10 monclient: _add_conns ranks=[1,2,0] 2024-03-13T07:34:53.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52400b1e70 0x7f5240139230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:53.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 10 monclient(hunting): picked mon.noname-b con 0x7f52400b1e70 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:53.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52400b2250 0x7f5240136280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:53.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 10 monclient(hunting): picked mon.noname-c con 0x7f52400b2250 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:53.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5240142d30 0x7f52401db310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:53.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 10 monclient(hunting): picked mon.noname-a con 0x7f5240142d30 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:53.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:53.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524014e7a0 con 0x7f5240142d30 2024-03-13T07:34:53.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:53.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52401433f0 con 0x7f52400b1e70 2024-03-13T07:34:53.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:53.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524014e920 con 0x7f52400b2250 2024-03-13T07:34:53.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:53.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.373+0000 7f52481b2640 10 monclient(hunting): authenticate will time out at 1320.782970s 2024-03-13T07:34:53.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f524674f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5240142d30 0x7f52401db310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:53.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f5245f4e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52400b1e70 0x7f5240139230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:53.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f524574d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52400b2250 0x7f5240136280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:53.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f524674f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5240142d30 0x7f52401db310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56518/0 (socket says 172.21.15.132:56518) 2024-03-13T07:34:53.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f524674f640 1 -- 172.21.15.132:0/961481605 learned_addr learned my addr 172.21.15.132:0/961481605 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:53.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f524674f640 10 monclient(hunting): get_auth_request con 0x7f5240142d30 auth_method 0 2024-03-13T07:34:53.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f524674f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:53.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f524674f640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:53.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f524674f640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:53.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f5245f4e640 10 monclient(hunting): get_auth_request con 0x7f52400b1e70 auth_method 0 2024-03-13T07:34:53.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f5245f4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:53.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f5245f4e640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:53.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f5245f4e640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:53.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f524574d640 10 monclient(hunting): get_auth_request con 0x7f52400b2250 auth_method 0 2024-03-13T07:34:53.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f524574d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:53.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f524574d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:53.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.374+0000 7f524574d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:53.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.375+0000 7f524674f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:53.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.375+0000 7f524674f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:53.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.375+0000 7f524674f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:53.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.375+0000 7f5245f4e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:53.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.375+0000 7f5245f4e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:53.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.375+0000 7f5245f4e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:53.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.375+0000 7f524574d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:53.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.375+0000 7f524574d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:53.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.375+0000 7f524574d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:53.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.377+0000 7f5245f4e640 10 monclient(hunting): handle_auth_done global_id 4279 payload 824 2024-03-13T07:34:53.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.377+0000 7f5245f4e640 1 -- 172.21.15.132:0/961481605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52400b2250 msgr2=0x7f5240136280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:53.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.377+0000 7f5245f4e640 1 --2- 172.21.15.132:0/961481605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52400b2250 0x7f5240136280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:53.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.377+0000 7f5245f4e640 1 -- 172.21.15.132:0/961481605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5240142d30 msgr2=0x7f52401db310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:53.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.377+0000 7f5245f4e640 1 --2- 172.21.15.132:0/961481605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5240142d30 0x7f52401db310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:53.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.377+0000 7f5245f4e640 10 monclient: _finish_hunting 0 2024-03-13T07:34:53.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.377+0000 7f5245f4e640 1 monclient: found mon.noname-b 2024-03-13T07:34:53.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.377+0000 7f5245f4e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:53.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.377+0000 7f5245f4e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:53.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.377+0000 7f5245f4e640 1 -- 172.21.15.132:0/961481605 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5240136d20 con 0x7f52400b1e70 2024-03-13T07:34:53.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.379+0000 7f5245f4e640 1 --2- 172.21.15.132:0/961481605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52400b1e70 0x7f5240139230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f523000c3c0 tx=0x7f5230001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:53.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.379+0000 7f52367fc640 10 client.?.objecter ms_handle_connect 0x7f52400b1e70 2024-03-13T07:34:53.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.379+0000 7f52367fc640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:53.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52367fc640 1 -- 172.21.15.132:0/961481605 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5230017070 con 0x7f52400b1e70 2024-03-13T07:34:53.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52367fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:53.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52367fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:53.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52367fc640 10 monclient: dump: 2024-03-13T07:34:53.386 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:53.386 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:53.386 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:53.386 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:53.386 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:53.386 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:53.386 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:53.386 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:53.386 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:53.387 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:53.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52367fc640 10 monclient: _finish_auth 0 2024-03-13T07:34:53.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52367fc640 10 monclient: _check_auth_tickets 2024-03-13T07:34:53.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:53.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52367fc640 1 -- 172.21.15.132:0/961481605 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5230005ce0 con 0x7f52400b1e70 2024-03-13T07:34:53.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52367fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:53.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52367fc640 1 -- 172.21.15.132:0/961481605 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52300072d0 con 0x7f52400b1e70 2024-03-13T07:34:53.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52481b2640 5 monclient: authenticate success, global_id 4279 2024-03-13T07:34:53.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52481b2640 10 monclient: _renew_subs 2024-03-13T07:34:53.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52481b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:53.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52481b2640 1 -- 172.21.15.132:0/961481605 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52401367c0 con 0x7f52400b1e70 2024-03-13T07:34:53.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f5237fff640 4 set_mon_vals no callback set 2024-03-13T07:34:53.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52367fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:53.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52367fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:53.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52367fc640 10 monclient: dump: 2024-03-13T07:34:53.388 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:53.388 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:53.388 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:53.388 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:53.388 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:53.388 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:53.388 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:53.388 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:53.388 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:53.388 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:53.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52481b2640 10 client.4279.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:53.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52481b2640 10 monclient: _renew_subs 2024-03-13T07:34:53.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52481b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:53.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52481b2640 1 -- 172.21.15.132:0/961481605 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52401db850 con 0x7f52400b1e70 2024-03-13T07:34:53.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.383+0000 7f52481b2640 1 librados: init done 2024-03-13T07:34:53.389 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.384+0000 7f52481b2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:53.389 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.384+0000 7f52481b2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:53.389 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.384+0000 7f52481b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:53.389 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.384+0000 7f52481b2640 1 -- 172.21.15.132:0/961481605 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524004f000 con 0x7f52400b1e70 2024-03-13T07:34:53.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.625+0000 7f52367fc640 1 -- 172.21.15.132:0/961481605 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f523001a070 con 0x7f52400b1e70 2024-03-13T07:34:53.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.626+0000 7f52367fc640 1 --2- 172.21.15.132:0/961481605 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f522406d560 0x7f522406fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:53.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.626+0000 7f524574d640 1 --2- 172.21.15.132:0/961481605 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f522406d560 0x7f522406fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:53.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.626+0000 7f524574d640 10 monclient: get_auth_request con 0x7f522406d560 auth_method 0 2024-03-13T07:34:53.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.626+0000 7f524574d640 1 --2- 172.21.15.132:0/961481605 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f522406d560 0x7f522406fa20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f523c036000 tx=0x7f523c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:53.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.626+0000 7f52367fc640 10 client.4279.objecter ms_handle_connect 0x7f522406d560 2024-03-13T07:34:53.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.628+0000 7f52367fc640 1 -- 172.21.15.132:0/961481605 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5230056780 con 0x7f52400b1e70 2024-03-13T07:34:53.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.628+0000 7f52367fc640 10 client.4279.objecter ms_dispatch 0x7f5240000b90 osd_map(25..25 src has 1..25) v4 2024-03-13T07:34:53.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.628+0000 7f52367fc640 3 client.4279.objecter handle_osd_map got epochs [25,25] > 0 2024-03-13T07:34:53.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.628+0000 7f52367fc640 3 client.4279.objecter handle_osd_map decoding full epoch 25 2024-03-13T07:34:53.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.628+0000 7f52367fc640 20 client.4279.objecter dump_active .. 0 homeless 2024-03-13T07:34:53.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.638+0000 7f52367fc640 1 -- 172.21.15.132:0/961481605 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f52300586e0 con 0x7f52400b1e70 2024-03-13T07:34:53.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.638+0000 7f52367fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:53.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.638+0000 7f52367fc640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:53.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.812+0000 7f52481b2640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:53.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.812+0000 7f52481b2640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:53.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.812+0000 7f52481b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:53.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.812+0000 7f52481b2640 1 -- 172.21.15.132:0/961481605 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52400af960 con 0x7f52400b1e70 2024-03-13T07:34:53.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.840+0000 7f52367fc640 1 -- 172.21.15.132:0/961481605 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+14766 (secure 0 0 0) 0x7f523005c780 con 0x7f52400b1e70 2024-03-13T07:34:53.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.840+0000 7f52367fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T07:34:53.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.840+0000 7f52367fc640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:53.841 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:34:53.841 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":25,"fsid":"74e4bc2f-6a5e-4423-a8c8-807970a2f96c","created":"2024-03-13T07:33:47.494293+0000","modified":"2024-03-13T07:34:52.782902+0000","last_up_change":"2024-03-13T07:34:44.296009+0000","last_in_change":"2024-03-13T07:33:53.171597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-13T07:34:33.504752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-13T07:34:49.707222+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0edb216c-bc1e-4865-aafe-218611f3eb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":590522438},{"type":"v1","addr":"172.21.15.132:6819","nonce":590522438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":590522438},{"type":"v1","addr":"172.21.15.132:6821","nonce":590522438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":590522438},{"type":"v1","addr":"172.21.15.132:6825","nonce":590522438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":590522438},{"type":"v1","addr":"172.21.15.132:6823","nonce":590522438}]},"public_addr":"172.21.15.132:6819/590522438","cluster_addr":"172.21.15.132:6821/590522438","heartbeat_back_addr":"172.21.15.132:6825/590522438","heartbeat_front_addr":"172.21.15.132:6823/590522438","state":["exists","up"]},{"osd":1,"uuid":"6be0ae6b-951e-4279-ba0a-9b05b6e6e85e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1071068206},{"type":"v1","addr":"172.21.15.132:6803","nonce":1071068206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1071068206},{"type":"v1","addr":"172.21.15.132:6805","nonce":1071068206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1071068206},{"type":"v1","addr":"172.21.15.132:6809","nonce":1071068206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1071068206},{"type":"v1","addr":"172.21.15.132:6807","nonce":1071068206}]},"public_addr":"172.21.15.132:6803/1071068206","cluster_addr":"172.21.15.132:6805/1071068206","heartbeat_back_addr":"172.21.15.132:6809/1071068206","heartbeat_front_addr":"172.21.15.132:6807/1071068206","state":["exists","up"]},{"osd":2,"uuid":"18008b7f-cfe7-4450-a3cf-aab63ebbe99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":958254560},{"type":"v1","addr":"172.21.15.132:6827","nonce":958254560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":958254560},{"type":"v1","addr":"172.21.15.132:6829","nonce":958254560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":958254560},{"type":"v1","addr":"172.21.15.132:6833","nonce":958254560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":958254560},{"type":"v1","addr":"172.21.15.132:6831","nonce":958254560}]},"public_addr":"172.21.15.132:6827/958254560","cluster_addr":"172.21.15.132:6829/958254560","heartbeat_back_addr":"172.21.15.132:6833/958254560","heartbeat_front_addr":"172.21.15.132:6831/958254560","state":["exists","up"]},{"osd":3,"uuid":"411b6066-4b9f-40e7-b92d-4ba914eab63e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6811","nonce":1770057764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6813","nonce":1770057764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6817","nonce":1770057764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6815","nonce":1770057764}]},"public_addr":"172.21.15.132:6811/1770057764","cluster_addr":"172.21.15.132:6813/1770057764","heartbeat_back_addr":"172.21.15.132:6817/1770057764","heartbeat_front_addr":"172.21.15.132:6815/1770057764","state":["exists","up"]},{"osd":4,"uuid":"1a719246-ae04-42b0-bd9c-fc318dc57ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":699638467},{"type":"v1","addr":"172.21.15.195:6803","nonce":699638467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":699638467},{"type":"v1","addr":"172.21.15.195:6809","nonce":699638467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":699638467},{"type":"v1","addr":"172.21.15.195:6815","nonce":699638467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6811","nonce":699638467},{"type":"v1","addr":"172.21.15.195:6813","nonce":699638467}]},"public_addr":"172.21.15.195:6803/699638467","cluster_addr":"172.21.15.195:6809/699638467","heartbeat_back_addr":"172.21.15.195:6815/699638467","heartbeat_front_addr":"172.21.15.195:6813/699638467","state":["exists","up"]},{"osd":5,"uuid":"9d9ee991-241b-4507-ba2f-e3b5e2844d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6817","nonce":1953907706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6819","nonce":1953907706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6825","nonce":1953907706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6821","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6822","nonce":1953907706}]},"public_addr":"172.21.15.195:6817/1953907706","cluster_addr":"172.21.15.195:6819/1953907706","heartbeat_back_addr":"172.21.15.195:6825/1953907706","heartbeat_front_addr":"172.21.15.195:6822/1953907706","state":["exists","up"]},{"osd":6,"uuid":"af8b34fd-927b-4860-971b-9fddf699a0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6801","nonce":4106887608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6805","nonce":4106887608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6812","nonce":4106887608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6808","nonce":4106887608}]},"public_addr":"172.21.15.195:6801/4106887608","cluster_addr":"172.21.15.195:6805/4106887608","heartbeat_back_addr":"172.21.15.195:6812/4106887608","heartbeat_front_addr":"172.21.15.195:6808/4106887608","state":["exists","up"]},{"osd":7,"uuid":"19a39645-8664-463b-bae3-d7aa1026e5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6823","nonce":3378300314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6827","nonce":3378300314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6831","nonce":3378300314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6829","nonce":3378300314}]},"public_addr":"172.21.15.195:6823/3378300314","cluster_addr":"172.21.15.195:6827/3378300314","heartbeat_back_addr":"172.21.15.195:6831/3378300314","heartbeat_front_addr":"172.21.15.195:6829/3378300314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:32.324299+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:31.786840+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:32.474925+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:31.877950+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:30.195463+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:30.243386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:30.409185+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:29.949919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T07:34:53.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.841+0000 7f52481b2640 10 librados: watch_flush enter 2024-03-13T07:34:53.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.841+0000 7f52481b2640 10 librados: watch_flush exit 2024-03-13T07:34:53.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.841+0000 7f52481b2640 20 client.4279.objecter shutdown clearing up homeless session... 2024-03-13T07:34:53.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.841+0000 7f52481b2640 10 client.4279.objecter successfully canceled tick 2024-03-13T07:34:53.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.841+0000 7f52481b2640 1 -- 172.21.15.132:0/961481605 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f522406d560 msgr2=0x7f522406fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:53.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.841+0000 7f52481b2640 1 --2- 172.21.15.132:0/961481605 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f522406d560 0x7f522406fa20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f523c036000 tx=0x7f523c008030 comp rx=0 tx=0).stop 2024-03-13T07:34:53.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.841+0000 7f52481b2640 10 monclient: shutdown 2024-03-13T07:34:53.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.841+0000 7f52481b2640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:53.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.841+0000 7f52481b2640 1 -- 172.21.15.132:0/961481605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52400b1e70 msgr2=0x7f5240139230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:53.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.841+0000 7f52481b2640 1 --2- 172.21.15.132:0/961481605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52400b1e70 0x7f5240139230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f523000c3c0 tx=0x7f5230001e70 comp rx=0 tx=0).stop 2024-03-13T07:34:53.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.841+0000 7f52481b2640 1 -- 172.21.15.132:0/961481605 shutdown_connections 2024-03-13T07:34:53.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.842+0000 7f52481b2640 1 --2- 172.21.15.132:0/961481605 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f522406d560 0x7f522406fa20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:53.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.842+0000 7f52481b2640 1 --2- 172.21.15.132:0/961481605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5240142d30 0x7f52401db310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:53.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.842+0000 7f52481b2640 1 --2- 172.21.15.132:0/961481605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52400b2250 0x7f5240136280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:53.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.842+0000 7f52481b2640 1 --2- 172.21.15.132:0/961481605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52400b1e70 0x7f5240139230 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:53.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.842+0000 7f52481b2640 1 -- 172.21.15.132:0/961481605 >> 172.21.15.132:0/961481605 conn(0x7f52400b7f70 msgr2=0x7f52400b51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:53.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.842+0000 7f52481b2640 1 -- 172.21.15.132:0/961481605 shutdown_connections 2024-03-13T07:34:53.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.842+0000 7f52481b2640 1 -- 172.21.15.132:0/961481605 wait complete. 2024-03-13T07:34:53.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:53.842+0000 7f52481b2640 1 librados: shutdown 2024-03-13T07:34:53.854 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-13T07:34:53.854 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-13T07:34:53.855 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-13T07:34:53.855 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-13T07:34:53.855 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-13T07:34:53.856 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-13T07:34:53.856 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-13T07:34:53.856 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-13T07:34:54.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.034+0000 7f4ff4205640 10 monclient: get_monmap_and_config 2024-03-13T07:34:54.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.034+0000 7f4ff4205640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.034+0000 7f4ff4205640 10 monclient: monmap: 2024-03-13T07:34:54.038 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.038 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.038 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:54.036028+0000 2024-03-13T07:34:54.038 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:54.036028+0000 2024-03-13T07:34:54.038 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.038 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.038 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:54.038 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:54.039 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:54.039 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:54.039 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:54.039 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.033+0000 7fcc9c9a3640 10 monclient: get_monmap_and_config 2024-03-13T07:34:54.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.033+0000 7fcc9c9a3640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.033+0000 7fcc9c9a3640 10 monclient: monmap: 2024-03-13T07:34:54.039 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.039 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.040 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:54.034997+0000 2024-03-13T07:34:54.040 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:54.034997+0000 2024-03-13T07:34:54.040 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.040 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.040 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:54.040 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:54.040 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:54.040 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:54.040 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:54.040 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.035+0000 7fcc9c9a3640 1 Processor -- start 2024-03-13T07:34:54.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.035+0000 7f4ff4205640 1 Processor -- start 2024-03-13T07:34:54.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.035+0000 7f4ff4205640 1 -- start start 2024-03-13T07:34:54.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.036+0000 7fcc9c9a3640 1 -- start start 2024-03-13T07:34:54.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.036+0000 7fcc9c9a3640 10 monclient: init 2024-03-13T07:34:54.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7fcc9c9a3640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7fcc9c9a3640 10 monclient: _add_conns ranks=[4,3,0,1,2] 2024-03-13T07:34:54.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7fcc9c9a3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcc90006fe0 0x7fcc900320d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7fcc9c9a3640 10 monclient(hunting): picked mon.noname-c con 0x7fcc90006fe0 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:54.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7fcc9c9a3640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcc90031630 0x7fcc900c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.035+0000 7f4ff4205640 10 monclient: init 2024-03-13T07:34:54.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7fcc9c9a3640 10 monclient(hunting): picked mon.noname-a con 0x7fcc90031630 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:54.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7fcc9c9a3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcc900073c0 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:54.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7fcc9c9a3640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc90033050 con 0x7fcc900073c0 2024-03-13T07:34:54.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7fcc9c9a3640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7fcc9c9a3640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc900c1650 con 0x7fcc90031630 2024-03-13T07:34:54.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7fcc9c9a3640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7fcc9c9a3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc90032610 con 0x7fcc90006fe0 2024-03-13T07:34:54.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7fcc9c9a3640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7fcc9c9a3640 10 monclient(hunting): authenticate will time out at 1321.446432s 2024-03-13T07:34:54.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.036+0000 7f4ff4205640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.036+0000 7f4ff4205640 10 monclient: _add_conns ranks=[3,2,0,4,1] 2024-03-13T07:34:54.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.036+0000 7f4ff4205640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4fec06e820 0x7f4fec06f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.036+0000 7f4ff4205640 10 monclient(hunting): picked mon.noname-a con 0x7f4fec06e820 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:54.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.036+0000 7f4ff4205640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06ec00 0x7f4fec13ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.036+0000 7f4ff4205640 10 monclient(hunting): picked mon.noname-b con 0x7f4fec06ec00 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.036+0000 7f4ff4205640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4fec13bf60 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:54.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.036+0000 7f4ff4205640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fec070a00 con 0x7f4fec13bf60 2024-03-13T07:34:54.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.036+0000 7f4ff4205640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.034+0000 7f3ce6cb7640 10 monclient: get_monmap_and_config 2024-03-13T07:34:54.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.036+0000 7f4ff4205640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fec070ce0 con 0x7f4fec06e820 2024-03-13T07:34:54.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.036+0000 7f4ff4205640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.036+0000 7f4ff4205640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec070fc0 con 0x7f4fec06ec00 2024-03-13T07:34:54.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.036+0000 7f4ff4205640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.036+0000 7f4ff4205640 10 monclient(hunting): authenticate will time out at 1321.445836s 2024-03-13T07:34:54.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.034+0000 7f3ce6cb7640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.034+0000 7f3ce6cb7640 10 monclient: monmap: 2024-03-13T07:34:54.046 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.046 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.047 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:54.036319+0000 2024-03-13T07:34:54.047 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:54.036319+0000 2024-03-13T07:34:54.047 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.047 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.047 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:54.047 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:54.047 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:54.047 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:54.047 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:54.047 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7f4ff17a0640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06ec00 0x7f4fec13ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7f4ff17a0640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06ec00 0x7f4fec13ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:42644/0 (socket says 172.21.15.132:42644) 2024-03-13T07:34:54.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7f4ff17a0640 1 -- 172.21.15.132:0/2719849051 learned_addr learned my addr 172.21.15.132:0/2719849051 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7f4ff17a0640 10 monclient(hunting): get_auth_request con 0x7f4fec06ec00 auth_method 0 2024-03-13T07:34:54.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7f4ff17a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7f4ff17a0640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7f4ff17a0640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7f3ce6cb7640 1 Processor -- start 2024-03-13T07:34:54.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7f3ce6cb7640 1 -- start start 2024-03-13T07:34:54.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7fcc977fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcc90006fe0 0x7fcc900320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7fcc977fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcc90006fe0 0x7fcc900320d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:48796/0 (socket says 172.21.15.132:48796) 2024-03-13T07:34:54.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7fcc977fe640 1 -- 172.21.15.132:0/29198020 learned_addr learned my addr 172.21.15.132:0/29198020 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7fcc977fe640 10 monclient(hunting): get_auth_request con 0x7fcc90006fe0 auth_method 0 2024-03-13T07:34:54.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7fcc977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7fcc977fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7fcc977fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.037+0000 7f3ce6cb7640 10 monclient: init 2024-03-13T07:34:54.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7f3ce6cb7640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7f3ce6cb7640 10 monclient: _add_conns ranks=[2,1,4,3,0] 2024-03-13T07:34:54.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7f3ce6cb7640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce008b4a0 0x7f3ce00aee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7f3ce6cb7640 10 monclient(hunting): picked mon.noname-b con 0x7f3ce008b4a0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7f3ce6cb7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3ce0152de0 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:54.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7f3ce6cb7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3ce01538d0 0x7f3ce00ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7f5087bac640 10 monclient: get_monmap_and_config 2024-03-13T07:34:54.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7f5087bac640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.038+0000 7f5087bac640 10 monclient: monmap: 2024-03-13T07:34:54.051 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.051 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.052 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:54.040203+0000 2024-03-13T07:34:54.052 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:54.040203+0000 2024-03-13T07:34:54.052 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.052 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.052 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:54.052 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:54.052 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:54.052 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:54.052 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:54.052 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7f3ce6cb7640 10 monclient(hunting): picked mon.noname-c con 0x7f3ce01538d0 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:54.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7f3ce6cb7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce00abae0 con 0x7f3ce0152de0 2024-03-13T07:34:54.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7f07acba3640 10 monclient: get_monmap_and_config 2024-03-13T07:34:54.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7f07acba3640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7f07acba3640 10 monclient: monmap: 2024-03-13T07:34:54.053 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.053 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.053 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:54.040893+0000 2024-03-13T07:34:54.053 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:54.040893+0000 2024-03-13T07:34:54.053 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.054 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.054 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:54.054 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:54.054 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:54.054 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:54.054 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:54.054 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7f3ce6cb7640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7f3ce6cb7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce00abdc0 con 0x7f3ce01538d0 2024-03-13T07:34:54.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7f3ce6cb7640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7f3ce6cb7640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00ac0a0 con 0x7f3ce008b4a0 2024-03-13T07:34:54.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7fcc977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7fcc977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7fcc977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7f3ce6cb7640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7f3ce6cb7640 10 monclient(hunting): authenticate will time out at 1321.448563s 2024-03-13T07:34:54.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7f4ff17a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7f4ff17a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.039+0000 7f4ff17a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.040+0000 7f5087bac640 1 Processor -- start 2024-03-13T07:34:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.040+0000 7f692e8f4640 10 monclient: get_monmap_and_config 2024-03-13T07:34:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.040+0000 7f692e8f4640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.040+0000 7f692e8f4640 10 monclient: monmap: 2024-03-13T07:34:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:54.042052+0000 2024-03-13T07:34:54.057 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:54.042052+0000 2024-03-13T07:34:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:54.058 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7fcc96ffd640 1 --2- 172.21.15.132:0/29198020 >> v2:172.21.15.132:3300/0 conn(0x7fcc90031630 0x7fcc900c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7fcc96ffd640 10 monclient(hunting): get_auth_request con 0x7fcc90031630 auth_method 0 2024-03-13T07:34:54.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7fcc96ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7fcc96ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7fcc96ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f07acba3640 1 Processor -- start 2024-03-13T07:34:54.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f3cdffff640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f3ce0152de0 0x7f3ce01531c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.132:47096/0 (socket says 172.21.15.132:47096) 2024-03-13T07:34:54.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f3cdffff640 1 -- 172.21.15.132:0/3786037963 learned_addr learned my addr 172.21.15.132:0/3786037963 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:34:54.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.040+0000 7f5087bac640 1 -- start start 2024-03-13T07:34:54.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.040+0000 7f5087bac640 10 monclient: init 2024-03-13T07:34:54.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f5087bac640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f5087bac640 10 monclient: _add_conns ranks=[0,2,1,3,4] 2024-03-13T07:34:54.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f5087bac640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f508006e820 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:54.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f5087bac640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508006ec00 0x7f508006f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f5087bac640 10 monclient(hunting): picked mon.noname-b con 0x7f508006ec00 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f5087bac640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f50800717c0 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:54.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f5087bac640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5080070a00 con 0x7f508006e820 2024-03-13T07:34:54.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f5087bac640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5080070ce0 con 0x7f50800717c0 2024-03-13T07:34:54.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f5087bac640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f5087bac640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5080070fc0 con 0x7f508006ec00 2024-03-13T07:34:54.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f5087bac640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f5087bac640 10 monclient(hunting): authenticate will time out at 1321.450525s 2024-03-13T07:34:54.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f07acba3640 1 -- start start 2024-03-13T07:34:54.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f07acba3640 10 monclient: init 2024-03-13T07:34:54.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7f692e8f4640 1 Processor -- start 2024-03-13T07:34:54.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f692e8f4640 1 -- start start 2024-03-13T07:34:54.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f07acba3640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f07acba3640 10 monclient: _add_conns ranks=[0,4,3,2,1] 2024-03-13T07:34:54.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f07acba3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f07a80a9980 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:54.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f07acba3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f07a80ae380 0x7f07a8149310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f07acba3640 10 monclient(hunting): picked mon.noname-c con 0x7f07a80ae380 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:54.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f07acba3640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f07a8149850 0x7f07a8151370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f07acba3640 10 monclient(hunting): picked mon.noname-a con 0x7f07a8149850 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:54.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f07acba3640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a80abe70 con 0x7f07a80a9980 2024-03-13T07:34:54.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f07acba3640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f07acba3640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a80a9d60 con 0x7f07a8149850 2024-03-13T07:34:54.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f07acba3640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f07acba3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a80ae100 con 0x7f07a80ae380 2024-03-13T07:34:54.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f07acba3640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f07acba3640 10 monclient(hunting): authenticate will time out at 1321.451512s 2024-03-13T07:34:54.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f692e8f4640 10 monclient: init 2024-03-13T07:34:54.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7fcc977fe640 10 monclient(hunting): handle_auth_done global_id 4282 payload 274 2024-03-13T07:34:54.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7fcc977fe640 1 -- 172.21.15.132:0/29198020 >> v2:172.21.15.132:3300/0 conn(0x7fcc90031630 msgr2=0x7fcc900c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7fcc977fe640 1 --2- 172.21.15.132:0/29198020 >> v2:172.21.15.132:3300/0 conn(0x7fcc90031630 0x7fcc900c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7fcc977fe640 1 -- 172.21.15.132:0/29198020 >> v1:172.21.15.132:6789/0 conn(0x7fcc900073c0 legacy=0x7fcc9002fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7fcc977fe640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7fcc977fe640 1 monclient: found mon.noname-c 2024-03-13T07:34:54.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7fcc977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7fcc977fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.041+0000 7fcc977fe640 1 -- 172.21.15.132:0/29198020 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc90033330 con 0x7fcc90006fe0 2024-03-13T07:34:54.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.042+0000 7f4ff17a0640 10 monclient(hunting): handle_auth_done global_id 4271 payload 274 2024-03-13T07:34:54.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.043+0000 7f4ff17a0640 1 -- 172.21.15.132:0/2719849051 >> v2:172.21.15.132:3300/0 conn(0x7f4fec06e820 msgr2=0x7f4fec06f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.043+0000 7f4ff17a0640 1 --2- 172.21.15.132:0/2719849051 >> v2:172.21.15.132:3300/0 conn(0x7f4fec06e820 0x7f4fec06f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.043+0000 7f4ff17a0640 1 -- 172.21.15.132:0/2719849051 >> v1:172.21.15.132:6789/0 conn(0x7f4fec13bf60 legacy=0x7f4fec0717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.043+0000 7f4ff17a0640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.043+0000 7f4ff17a0640 1 monclient: found mon.noname-b 2024-03-13T07:34:54.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.043+0000 7f4ff17a0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.043+0000 7f4ff17a0640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.043+0000 7f4ff17a0640 1 -- 172.21.15.132:0/2719849051 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec06fe20 con 0x7f4fec06ec00 2024-03-13T07:34:54.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.043+0000 7f692e8f4640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.043+0000 7f692e8f4640 10 monclient: _add_conns ranks=[1,0,3,2,4] 2024-03-13T07:34:54.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.043+0000 7f692e8f4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6928072270 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:54.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.043+0000 7f692e8f4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f692806fc70 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:54.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.043+0000 7f692e8f4640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f69280ae620 0x7f69280aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.043+0000 7f692e8f4640 10 monclient(hunting): picked mon.noname-a con 0x7f69280ae620 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:54.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.043+0000 7f692e8f4640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6928070c00 con 0x7f692806fc70 2024-03-13T07:34:54.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.043+0000 7f692e8f4640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692806f750 con 0x7f6928072270 2024-03-13T07:34:54.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.044+0000 7f692e8f4640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.044+0000 7f692e8f4640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69280706e0 con 0x7f69280ae620 2024-03-13T07:34:54.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.044+0000 7f692e8f4640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.044+0000 7f692e8f4640 10 monclient(hunting): authenticate will time out at 1321.453165s 2024-03-13T07:34:54.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.044+0000 7f5085147640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508006ec00 0x7f508006f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.044+0000 7f5085147640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508006ec00 0x7f508006f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:42664/0 (socket says 172.21.15.132:42664) 2024-03-13T07:34:54.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.044+0000 7f5085147640 1 -- 172.21.15.132:0/3933159473 learned_addr learned my addr 172.21.15.132:0/3933159473 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.044+0000 7f5085147640 10 monclient(hunting): get_auth_request con 0x7f508006ec00 auth_method 0 2024-03-13T07:34:54.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.044+0000 7f5085147640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.044+0000 7f5085147640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.044+0000 7f5085147640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.044+0000 7fb8959bb640 10 monclient: get_monmap_and_config 2024-03-13T07:34:54.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.045+0000 7fb8959bb640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.045+0000 7fb8959bb640 10 monclient: monmap: 2024-03-13T07:34:54.068 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.068 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.068 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:54.046779+0000 2024-03-13T07:34:54.069 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:54.046779+0000 2024-03-13T07:34:54.069 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.069 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.069 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:54.069 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:54.069 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:54.069 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:54.069 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:54.069 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.045+0000 7f3ce4a53640 1 --2- 172.21.15.132:0/3786037963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce008b4a0 0x7f3ce00aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 1 Processor -- start 2024-03-13T07:34:54.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 1 -- start start 2024-03-13T07:34:54.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 10 monclient: init 2024-03-13T07:34:54.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fcc977fe640 1 --2- 172.21.15.132:0/29198020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc90006fe0 0x7fcc900320d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc900077b0 tx=0x7fcc8c02b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4216dd9b4e03aa31 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7f3ce5254640 1 --2- 172.21.15.132:0/3786037963 >> v2:172.21.15.195:3300/0 conn(0x7f3ce01538d0 0x7f3ce00ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2024-03-13T07:34:54.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb888037510 0x7fb8880c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 10 monclient(hunting): picked mon.noname-a con 0x7fb888037510 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:54.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb888035aa0 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:54.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb888030a80 0x7fb8880cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 10 monclient(hunting): picked mon.noname-b con 0x7fb888030a80 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb888033db0 con 0x7fb888035aa0 2024-03-13T07:34:54.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8880c2310 con 0x7fb888037510 2024-03-13T07:34:54.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880378f0 con 0x7fb888030a80 2024-03-13T07:34:54.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.046+0000 7fb8959bb640 10 monclient(hunting): authenticate will time out at 1321.456064s 2024-03-13T07:34:54.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.047+0000 7f3ce5254640 10 monclient(hunting): get_auth_request con 0x7f3ce01538d0 auth_method 0 2024-03-13T07:34:54.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.047+0000 7f3ce5254640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.047+0000 7f3ce5254640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.047+0000 7f3ce5254640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.047+0000 7f07a7fff640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f07a8149850 0x7f07a8151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.047+0000 7f3ce4a53640 10 monclient(hunting): get_auth_request con 0x7f3ce008b4a0 auth_method 0 2024-03-13T07:34:54.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.047+0000 7f3ce4a53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.047+0000 7f3ce4a53640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.047+0000 7f3ce4a53640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.048+0000 7f07a7fff640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f07a8149850 0x7f07a8151370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56540/0 (socket says 172.21.15.132:56540) 2024-03-13T07:34:54.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.048+0000 7f07a7fff640 1 -- 172.21.15.132:0/1617044316 learned_addr learned my addr 172.21.15.132:0/1617044316 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.048+0000 7f07a7fff640 10 monclient(hunting): get_auth_request con 0x7f07a8149850 auth_method 0 2024-03-13T07:34:54.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.048+0000 7f07a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.048+0000 7f07a7fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.048+0000 7f4ff17a0640 1 --2- 172.21.15.132:0/2719849051 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06ec00 0x7f4fec13ba20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8008870 tx=0x7f4fe8030080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4018ccddb801d8df server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.075 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.048+0000 7f07a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.075 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.049+0000 7f07a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.075 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.049+0000 7f07a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.075 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.049+0000 7f07a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.075 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.049+0000 7f5085147640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.076 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.049+0000 7f5085147640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.076 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.049+0000 7f5085147640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.076 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.049+0000 7f07a6ffd640 1 --2- 172.21.15.132:0/1617044316 >> v2:172.21.15.195:3300/0 conn(0x7f07a80ae380 0x7f07a8149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.050+0000 7f3ce5254640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.050+0000 7f3ce5254640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.050+0000 7f3ce5254640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.050+0000 7f3ce4a53640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.050+0000 7f3ce4a53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.050+0000 7f3ce4a53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.050+0000 7f3cdf7fe640 1 -- 172.21.15.132:0/3786037963 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074890188 0 0) 0x7f3ce00abae0 con 0x7f3ce0152de0 2024-03-13T07:34:54.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.050+0000 7f3cdf7fe640 10 monclient(hunting): _negotiate 2024-03-13T07:34:54.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.050+0000 7f3cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.050+0000 7f3cdf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.050+0000 7f3cdf7fe640 10 monclient(hunting): my global_id is 4285 2024-03-13T07:34:54.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.050+0000 7f3cdf7fe640 1 -- 172.21.15.132:0/3786037963 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cc8003520 con 0x7f3ce0152de0 2024-03-13T07:34:54.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.050+0000 7f07a6ffd640 10 monclient(hunting): get_auth_request con 0x7f07a80ae380 auth_method 0 2024-03-13T07:34:54.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.050+0000 7f07a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.050+0000 7f07a6ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.050+0000 7f07a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.051+0000 7f07a7fff640 10 monclient(hunting): handle_auth_done global_id 4278 payload 274 2024-03-13T07:34:54.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.051+0000 7f07a7fff640 1 -- 172.21.15.132:0/1617044316 >> v2:172.21.15.195:3300/0 conn(0x7f07a80ae380 msgr2=0x7f07a8149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.051+0000 7f07a7fff640 1 --2- 172.21.15.132:0/1617044316 >> v2:172.21.15.195:3300/0 conn(0x7f07a80ae380 0x7f07a8149310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.051+0000 7f07a7fff640 1 -- 172.21.15.132:0/1617044316 >> v1:172.21.15.132:6789/0 conn(0x7f07a80a9980 legacy=0x7f07a80afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.051+0000 7f07a7fff640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.051+0000 7f07a7fff640 1 monclient: found mon.noname-a 2024-03-13T07:34:54.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.051+0000 7f07a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.051+0000 7f07a7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.051+0000 7f07a7fff640 1 -- 172.21.15.132:0/1617044316 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a80ae7c0 con 0x7f07a8149850 2024-03-13T07:34:54.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f855d24b640 10 monclient: get_monmap_and_config 2024-03-13T07:34:54.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f855d24b640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f855d24b640 10 monclient: monmap: 2024-03-13T07:34:54.080 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.080 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.080 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:54.053898+0000 2024-03-13T07:34:54.080 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:54.053898+0000 2024-03-13T07:34:54.080 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.080 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.080 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:54.080 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:54.080 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:54.080 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:54.080 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:54.080 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f3ce5254640 10 monclient(hunting): handle_auth_done global_id 4288 payload 274 2024-03-13T07:34:54.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f3ce5254640 1 -- 172.21.15.132:0/3786037963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce008b4a0 msgr2=0x7f3ce00aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f3ce5254640 1 --2- 172.21.15.132:0/3786037963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce008b4a0 0x7f3ce00aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f3ce5254640 1 -- 172.21.15.132:0/3786037963 >> v1:172.21.15.195:6789/0 conn(0x7f3ce0152de0 legacy=0x7f3ce01531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f3ce5254640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f3ce5254640 1 monclient: found mon.noname-c 2024-03-13T07:34:54.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f3ce5254640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f3ce5254640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f3ce5254640 1 -- 172.21.15.132:0/3786037963 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce00af3d0 con 0x7f3ce01538d0 2024-03-13T07:34:54.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f5085147640 10 monclient(hunting): handle_auth_done global_id 4274 payload 274 2024-03-13T07:34:54.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f5085147640 1 -- 172.21.15.132:0/3933159473 >> v1:172.21.15.195:6789/0 conn(0x7f50800717c0 legacy=0x7f5080071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f5085147640 1 -- 172.21.15.132:0/3933159473 >> v1:172.21.15.132:6789/0 conn(0x7f508006e820 legacy=0x7f508013ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f5085147640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f5085147640 1 monclient: found mon.noname-b 2024-03-13T07:34:54.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f5085147640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f5085147640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.052+0000 7f5085147640 1 -- 172.21.15.132:0/3933159473 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508006fe20 con 0x7f508006ec00 2024-03-13T07:34:54.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.053+0000 7f855d24b640 1 Processor -- start 2024-03-13T07:34:54.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.053+0000 7f855d24b640 1 -- start start 2024-03-13T07:34:54.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.053+0000 7f855d24b640 10 monclient: init 2024-03-13T07:34:54.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.054+0000 7f855d24b640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.054+0000 7f855d24b640 10 monclient: _add_conns ranks=[3,1,0,2,4] 2024-03-13T07:34:54.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.054+0000 7f855d24b640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f85580b53e0 0x7f85580b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.054+0000 7f855d24b640 10 monclient(hunting): picked mon.noname-a con 0x7f85580b53e0 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:54.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.054+0000 7f855d24b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f855813f950 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:54.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.054+0000 7f855d24b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f85580b0270 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:54.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.054+0000 7f855d24b640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85580b1570 con 0x7f85580b0270 2024-03-13T07:34:54.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.054+0000 7f855d24b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85580b27b0 con 0x7f855813f950 2024-03-13T07:34:54.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.054+0000 7f855d24b640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.054+0000 7f855d24b640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85580b57c0 con 0x7f85580b53e0 2024-03-13T07:34:54.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.054+0000 7f855d24b640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.054+0000 7f855d24b640 10 monclient(hunting): authenticate will time out at 1321.463620s 2024-03-13T07:34:54.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc967fc640 1 -- 172.21.15.132:0/29198020 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c02e070 con 0x7fcc90006fe0 2024-03-13T07:34:54.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc967fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:54.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc967fc640 10 monclient: dump: 2024-03-13T07:34:54.086 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.086 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.086 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.086 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.086 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.086 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.086 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.086 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.086 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.086 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc967fc640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc967fc640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc967fc640 1 -- 172.21.15.132:0/29198020 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8c037070 con 0x7fcc90006fe0 2024-03-13T07:34:54.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc9c9a3640 5 monclient: authenticate success, global_id 4282 2024-03-13T07:34:54.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc9c9a3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:54.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc967fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc967fc640 1 -- 172.21.15.132:0/29198020 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c031a00 con 0x7fcc90006fe0 2024-03-13T07:34:54.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc967fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:54.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc967fc640 10 monclient: dump: 2024-03-13T07:34:54.087 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.087 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.087 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.087 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.088 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.088 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.088 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.088 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.088 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.088 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc9c9a3640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:54.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.057+0000 7fcc9c9a3640 4 set_mon_vals no callback set 2024-03-13T07:34:54.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.058+0000 7fb88ffff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fb888035aa0 0x7fb888030370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:36026/0 (socket says 172.21.15.132:36026) 2024-03-13T07:34:54.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.058+0000 7fb88ffff640 1 -- 172.21.15.132:0/2611773776 learned_addr learned my addr 172.21.15.132:0/2611773776 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:34:54.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.058+0000 7f3ce5254640 1 --2- 172.21.15.132:0/3786037963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ce01538d0 0x7f3ce00ad4d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009d90 tx=0x7f3cd802be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c590e97c4386bdb server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.058+0000 7fcc9c9a3640 10 monclient: shutdown 2024-03-13T07:34:54.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.058+0000 7fcc9c9a3640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.058+0000 7fcc9c9a3640 1 -- 172.21.15.132:0/29198020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc90006fe0 msgr2=0x7fcc900320d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.058+0000 7f07a7fff640 1 --2- 172.21.15.132:0/1617044316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07a8149850 0x7f07a8151370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f07a806db60 tx=0x7f07a002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e796ffe137619e server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.058+0000 7fcc9c9a3640 1 --2- 172.21.15.132:0/29198020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc90006fe0 0x7fcc900320d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc900077b0 tx=0x7fcc8c02b880 comp rx=0 tx=0).stop 2024-03-13T07:34:54.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.058+0000 7fcc9c9a3640 1 -- 172.21.15.132:0/29198020 shutdown_connections 2024-03-13T07:34:54.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.058+0000 7fcc9c9a3640 1 --2- 172.21.15.132:0/29198020 >> v2:172.21.15.132:3300/0 conn(0x7fcc90031630 0x7fcc900c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.058+0000 7fcc9c9a3640 1 --2- 172.21.15.132:0/29198020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc90006fe0 0x7fcc900320d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.058+0000 7fcc9c9a3640 1 -- 172.21.15.132:0/29198020 >> 172.21.15.132:0/29198020 conn(0x7fcc90006070 msgr2=0x7fcc90006480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.058+0000 7fcc9c9a3640 1 -- 172.21.15.132:0/29198020 shutdown_connections 2024-03-13T07:34:54.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.058+0000 7fcc9c9a3640 1 -- 172.21.15.132:0/29198020 wait complete. 2024-03-13T07:34:54.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.058+0000 7f692e0f3640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f69280ae620 0x7f69280aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.059+0000 7fcc9c9a3640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.059+0000 7fcc9c9a3640 10 monclient: monmap: 2024-03-13T07:34:54.091 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.091 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:54.091 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:54.091 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:54.091 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.092 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.092 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:54.092 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:54.092 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:54.092 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.059+0000 7fcc9c9a3640 1 librados: starting msgr at 2024-03-13T07:34:54.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.059+0000 7fcc9c9a3640 1 librados: starting objecter 2024-03-13T07:34:54.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.059+0000 7f854ffff640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f855813f950 0x7f85580afb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.132:47138/0 (socket says 172.21.15.132:47138) 2024-03-13T07:34:54.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.059+0000 7f854ffff640 1 -- 172.21.15.132:0/1711219165 learned_addr learned my addr 172.21.15.132:0/1711219165 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:34:54.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.059+0000 7fcc9c9a3640 1 Processor -- start 2024-03-13T07:34:54.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f692e0f3640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f69280ae620 0x7f69280aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56546/0 (socket says 172.21.15.132:56546) 2024-03-13T07:34:54.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f692e0f3640 1 -- 172.21.15.132:0/2683855404 learned_addr learned my addr 172.21.15.132:0/2683855404 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f692e0f3640 10 monclient(hunting): get_auth_request con 0x7f69280ae620 auth_method 0 2024-03-13T07:34:54.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f692e0f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f692e0f3640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f692e0f3640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7fcc9c9a3640 1 -- start start 2024-03-13T07:34:54.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7fcc9c9a3640 1 librados: setting wanted keys 2024-03-13T07:34:54.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7fcc9c9a3640 1 librados: calling monclient init 2024-03-13T07:34:54.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7fcc9c9a3640 10 monclient: init 2024-03-13T07:34:54.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f4ff0f9f640 1 -- 172.21.15.132:0/2719849051 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8036070 con 0x7f4fec06ec00 2024-03-13T07:34:54.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f4ff0f9f640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f4ff0f9f640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:54.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f4ff0f9f640 10 monclient: dump: 2024-03-13T07:34:54.095 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.096 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.096 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.096 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.096 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.096 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.096 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.096 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.096 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.096 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f4ff0f9f640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f4ff0f9f640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f4ff0f9f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f4ff0f9f640 1 -- 172.21.15.132:0/2719849051 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe8030e30 con 0x7f4fec06ec00 2024-03-13T07:34:54.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f4ff0f9f640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f4ff0f9f640 1 -- 172.21.15.132:0/2719849051 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe803a600 con 0x7f4fec06ec00 2024-03-13T07:34:54.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f4ff0f9f640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f4ff0f9f640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:54.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f4ff0f9f640 10 monclient: dump: 2024-03-13T07:34:54.097 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.097 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.097 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.097 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.097 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.097 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.097 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.097 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.097 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.097 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f4ff4205640 5 monclient: authenticate success, global_id 4271 2024-03-13T07:34:54.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f4ff4205640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:54.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7f4ff4205640 4 set_mon_vals no callback set 2024-03-13T07:34:54.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7fcc9c9a3640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7fcc9c9a3640 10 monclient: _add_conns ranks=[0,1,2] 2024-03-13T07:34:54.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7fb8951ba640 1 --2- 172.21.15.132:0/2611773776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb888030a80 0x7fb8880cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.060+0000 7fcc9c9a3640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc90006fe0 0x7fcc9002a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7fcc9c9a3640 10 monclient(hunting): picked mon.noname-a con 0x7fcc90006fe0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7fcc9c9a3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc90031630 0x7fcc900c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7fcc9c9a3640 10 monclient(hunting): picked mon.noname-b con 0x7fcc90031630 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7fcc9c9a3640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc90029690 0x7fcc900b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7fcc9c9a3640 10 monclient(hunting): picked mon.noname-c con 0x7fcc90029690 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7fcc9c9a3640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7fcc9c9a3640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900c76e0 con 0x7fcc90006fe0 2024-03-13T07:34:54.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7fcc9c9a3640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7fcc9c9a3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc90031c40 con 0x7fcc90031630 2024-03-13T07:34:54.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7fcc9c9a3640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7fcc9c9a3640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc90006b90 con 0x7fcc90029690 2024-03-13T07:34:54.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7fcc9c9a3640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7fcc9c9a3640 10 monclient(hunting): authenticate will time out at 1321.470595s 2024-03-13T07:34:54.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7f4ff4205640 10 monclient: shutdown 2024-03-13T07:34:54.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7f4ff4205640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7f4ff4205640 1 -- 172.21.15.132:0/2719849051 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06ec00 msgr2=0x7f4fec13ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7f4ff4205640 1 --2- 172.21.15.132:0/2719849051 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06ec00 0x7f4fec13ba20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8008870 tx=0x7f4fe8030080 comp rx=0 tx=0).stop 2024-03-13T07:34:54.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7f4ff4205640 1 -- 172.21.15.132:0/2719849051 shutdown_connections 2024-03-13T07:34:54.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7f4ff4205640 1 --2- 172.21.15.132:0/2719849051 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec06ec00 0x7f4fec13ba20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7f4ff4205640 1 --2- 172.21.15.132:0/2719849051 >> v2:172.21.15.132:3300/0 conn(0x7f4fec06e820 0x7f4fec06f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7f4ff4205640 1 -- 172.21.15.132:0/2719849051 >> 172.21.15.132:0/2719849051 conn(0x7f4fec0b3c40 msgr2=0x7f4fec0b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7f4ff4205640 1 -- 172.21.15.132:0/2719849051 shutdown_connections 2024-03-13T07:34:54.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.061+0000 7f4ff4205640 1 -- 172.21.15.132:0/2719849051 wait complete. 2024-03-13T07:34:54.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.062+0000 7fb8951ba640 10 monclient(hunting): get_auth_request con 0x7fb888030a80 auth_method 0 2024-03-13T07:34:54.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.062+0000 7fb8951ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.062+0000 7fb8951ba640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.062+0000 7fb8951ba640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.062+0000 7f4ff4205640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.062+0000 7f5085147640 1 --2- 172.21.15.132:0/3933159473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508006ec00 0x7f508006f820 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5078009730 tx=0x7f507802e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c00cedf1a182b1cb server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.062+0000 7f4ff4205640 10 monclient: monmap: 2024-03-13T07:34:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:54.105 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.062+0000 7f4ff4205640 1 librados: starting msgr at 2024-03-13T07:34:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.062+0000 7f4ff4205640 1 librados: starting objecter 2024-03-13T07:34:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 1 Processor -- start 2024-03-13T07:34:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 1 -- start start 2024-03-13T07:34:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 1 librados: setting wanted keys 2024-03-13T07:34:54.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 1 librados: calling monclient init 2024-03-13T07:34:54.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 10 monclient: init 2024-03-13T07:34:54.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T07:34:54.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fec06e820 0x7f4fec140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7fb8951ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7fb8951ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7fb8951ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 10 monclient(hunting): picked mon.noname-a con 0x7f4fec06e820 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec13bf60 0x7f4fec13f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 10 monclient(hunting): picked mon.noname-c con 0x7f4fec13bf60 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fec133fb0 0x7f4fec134460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 10 monclient(hunting): picked mon.noname-b con 0x7f4fec133fb0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0b48e0 con 0x7f4fec06e820 2024-03-13T07:34:54.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0b4760 con 0x7f4fec133fb0 2024-03-13T07:34:54.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec071ed0 con 0x7f4fec13bf60 2024-03-13T07:34:54.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f4ff4205640 10 monclient(hunting): authenticate will time out at 1321.473029s 2024-03-13T07:34:54.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.063+0000 7f692e0f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.064+0000 7f692e0f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.064+0000 7f692e0f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.064+0000 7fb88f7fe640 1 -- 172.21.15.132:0/2611773776 <== mon.0 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925675448 0 0) 0x7fb888033db0 con 0x7fb888035aa0 2024-03-13T07:34:54.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.064+0000 7fb88f7fe640 10 monclient(hunting): _negotiate 2024-03-13T07:34:54.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.064+0000 7fb88f7fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.064+0000 7fb88f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.064+0000 7fb88f7fe640 10 monclient(hunting): my global_id is 4281 2024-03-13T07:34:54.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.064+0000 7fb88f7fe640 1 -- 172.21.15.132:0/2611773776 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb880005730 con 0x7fb888035aa0 2024-03-13T07:34:54.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.068+0000 7fcc96ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc90031630 0x7fcc900c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.068+0000 7fb88f7fe640 1 -- 172.21.15.132:0/2611773776 <== mon.0 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 823525922 0 0) 0x7fb880005730 con 0x7fb888035aa0 2024-03-13T07:34:54.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.068+0000 7fb88f7fe640 1 -- 172.21.15.132:0/2611773776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb888030a80 msgr2=0x7fb8880cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.068+0000 7fb88f7fe640 1 --2- 172.21.15.132:0/2611773776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb888030a80 0x7fb8880cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.068+0000 7fb88f7fe640 1 -- 172.21.15.132:0/2611773776 >> v2:172.21.15.132:3300/0 conn(0x7fb888037510 msgr2=0x7fb8880c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.068+0000 7fb88f7fe640 1 --2- 172.21.15.132:0/2611773776 >> v2:172.21.15.132:3300/0 conn(0x7fb888037510 0x7fb8880c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.068+0000 7fb88f7fe640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.068+0000 7fb88f7fe640 1 monclient: found mon.noname-a-legacy 2024-03-13T07:34:54.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.068+0000 7fb88f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.068+0000 7fb88f7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.132:6789/0 2024-03-13T07:34:54.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.068+0000 7fb88f7fe640 1 -- 172.21.15.132:0/2611773776 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb888034020 con 0x7fb888035aa0 2024-03-13T07:34:54.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.068+0000 7fb88f7fe640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.068+0000 7fb88f7fe640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.068+0000 7fb88f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 1 -- 172.21.15.132:0/2611773776 <== mon.0 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1639563336 0 0) 0x7fb87c001670 con 0x7fb888035aa0 2024-03-13T07:34:54.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-03-13T07:34:54.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 10 monclient: dump: 2024-03-13T07:34:54.112 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.112 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.112 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.112 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.112 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.112 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.112 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.113 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.113 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.113 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] but i'm connected to v1:172.21.15.132:6789/0, reconnecting 2024-03-13T07:34:54.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 1 -- 172.21.15.132:0/2611773776 >> v1:172.21.15.132:6789/0 conn(0x7fb888035aa0 legacy=0x7fb888030370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 10 monclient: _add_conns ranks=[1,2,0] 2024-03-13T07:34:54.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 1 --2- 172.21.15.132:0/2611773776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb880005bc0 0x7fb880005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 10 monclient(hunting): picked mon.b con 0x7fb880005bc0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 1 --2- 172.21.15.132:0/2611773776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb888030a80 0x7fb880006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 10 monclient(hunting): picked mon.c con 0x7fb888030a80 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 1 --2- 172.21.15.132:0/2611773776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb880006f50 0x7fb8800073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 10 monclient(hunting): picked mon.a con 0x7fb880006f50 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 1 -- 172.21.15.132:0/2611773776 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880c2310 con 0x7fb880006f50 2024-03-13T07:34:54.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 1 -- 172.21.15.132:0/2611773776 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880378f0 con 0x7fb880005bc0 2024-03-13T07:34:54.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 1 -- 172.21.15.132:0/2611773776 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88000dd50 con 0x7fb888030a80 2024-03-13T07:34:54.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7fb88f7fe640 10 monclient(hunting): _finish_auth 0 2024-03-13T07:34:54.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7f3cdf7fe640 1 -- 172.21.15.132:0/3786037963 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8039070 con 0x7f3ce01538d0 2024-03-13T07:34:54.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7f3cdf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7f3cdf7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:54.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7f3cdf7fe640 10 monclient: dump: 2024-03-13T07:34:54.116 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.116 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.116 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.116 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.116 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.116 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.116 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.116 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.116 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.116 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7f3cdf7fe640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7f3cdf7fe640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7f3cdf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7f3cdf7fe640 1 -- 172.21.15.132:0/3786037963 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd802cbf0 con 0x7f3ce01538d0 2024-03-13T07:34:54.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7f3cdf7fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7f3cdf7fe640 1 -- 172.21.15.132:0/3786037963 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8034440 con 0x7f3ce01538d0 2024-03-13T07:34:54.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7f3cdf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7f3cdf7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:54.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7f3cdf7fe640 10 monclient: dump: 2024-03-13T07:34:54.117 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.117 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.117 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.117 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.117 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.117 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.117 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.117 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.117 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.117 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7f3ce6cb7640 5 monclient: authenticate success, global_id 4288 2024-03-13T07:34:54.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7f3ce6cb7640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:54.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.069+0000 7f3ce6cb7640 4 set_mon_vals no callback set 2024-03-13T07:34:54.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.070+0000 7f3ce6cb7640 10 monclient: shutdown 2024-03-13T07:34:54.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.070+0000 7f3ce6cb7640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.070+0000 7f3ce6cb7640 1 -- 172.21.15.132:0/3786037963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ce01538d0 msgr2=0x7f3ce00ad4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.070+0000 7f3ce6cb7640 1 --2- 172.21.15.132:0/3786037963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ce01538d0 0x7f3ce00ad4d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009d90 tx=0x7f3cd802be40 comp rx=0 tx=0).stop 2024-03-13T07:34:54.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.070+0000 7f3ce6cb7640 1 -- 172.21.15.132:0/3786037963 shutdown_connections 2024-03-13T07:34:54.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.070+0000 7f3ce6cb7640 1 --2- 172.21.15.132:0/3786037963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ce01538d0 0x7f3ce00ad4d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.070+0000 7f3ce6cb7640 1 --2- 172.21.15.132:0/3786037963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce008b4a0 0x7f3ce00aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.070+0000 7f3ce6cb7640 1 -- 172.21.15.132:0/3786037963 >> 172.21.15.132:0/3786037963 conn(0x7f3ce00b0660 msgr2=0x7f3ce00b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.070+0000 7f3ce6cb7640 1 -- 172.21.15.132:0/3786037963 shutdown_connections 2024-03-13T07:34:54.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.070+0000 7fcc96ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc90031630 0x7fcc900c1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:48826/0 (socket says 172.21.15.132:48826) 2024-03-13T07:34:54.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.070+0000 7f3ce6cb7640 1 -- 172.21.15.132:0/3786037963 wait complete. 2024-03-13T07:34:54.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.070+0000 7fcc96ffd640 1 -- 172.21.15.132:0/3861919449 learned_addr learned my addr 172.21.15.132:0/3861919449 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.070+0000 7fcc96ffd640 10 monclient(hunting): get_auth_request con 0x7fcc90031630 auth_method 0 2024-03-13T07:34:54.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.070+0000 7fcc96ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.070+0000 7fcc96ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.070+0000 7fcc96ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.071+0000 7f3ce6cb7640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.071+0000 7f3ce6cb7640 10 monclient: monmap: 2024-03-13T07:34:54.120 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.120 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:54.121 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:54.121 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:54.121 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.121 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.121 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:54.121 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:54.121 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:54.121 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.071+0000 7fcc96ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.071+0000 7fcc96ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.071+0000 7fcc96ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.071+0000 7f3ce6cb7640 1 librados: starting msgr at 2024-03-13T07:34:54.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.071+0000 7f3ce6cb7640 1 librados: starting objecter 2024-03-13T07:34:54.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.071+0000 7f3ce6cb7640 1 Processor -- start 2024-03-13T07:34:54.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.071+0000 7fcc9dc05640 1 --2- 172.21.15.132:0/3861919449 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc90029690 0x7fcc900b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.071+0000 7f3ce6cb7640 1 -- start start 2024-03-13T07:34:54.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.071+0000 7f3ce6cb7640 1 librados: setting wanted keys 2024-03-13T07:34:54.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.071+0000 7f3ce6cb7640 1 librados: calling monclient init 2024-03-13T07:34:54.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.071+0000 7f3ce6cb7640 10 monclient: init 2024-03-13T07:34:54.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.072+0000 7f3ce6cb7640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.072+0000 7f3ce6cb7640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T07:34:54.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.072+0000 7f3ce6cb7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ce008b4a0 0x7f3ce00b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.072+0000 7f3ce6cb7640 10 monclient(hunting): picked mon.noname-a con 0x7f3ce008b4a0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.072+0000 7f3ce6cb7640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce0152de0 0x7f3ce006f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.072+0000 7f3ce6cb7640 10 monclient(hunting): picked mon.noname-c con 0x7f3ce0152de0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.072+0000 7f3ce6cb7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ce01538d0 0x7f3ce006fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.072+0000 7f3ce6cb7640 10 monclient(hunting): picked mon.noname-b con 0x7f3ce01538d0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.072+0000 7f3ce6cb7640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.072+0000 7f3ce6cb7640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00ab0a0 con 0x7f3ce008b4a0 2024-03-13T07:34:54.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.072+0000 7f3ce6cb7640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.072+0000 7f3ce6cb7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00aaf20 con 0x7f3ce01538d0 2024-03-13T07:34:54.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.072+0000 7f3ce6cb7640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.072+0000 7f3ce6cb7640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00ada10 con 0x7f3ce0152de0 2024-03-13T07:34:54.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.072+0000 7f3ce6cb7640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.072+0000 7f3ce6cb7640 10 monclient(hunting): authenticate will time out at 1321.481694s 2024-03-13T07:34:54.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.073+0000 7f8556d9d640 1 --2- 172.21.15.132:0/1711219165 >> v2:172.21.15.132:3300/0 conn(0x7f85580b53e0 0x7f85580b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.073+0000 7fcc9dc05640 10 monclient(hunting): get_auth_request con 0x7fcc90029690 auth_method 0 2024-03-13T07:34:54.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.073+0000 7fcc9dc05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.073+0000 7fcc9dc05640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.073+0000 7fcc9dc05640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.074+0000 7f8556d9d640 10 monclient(hunting): get_auth_request con 0x7f85580b53e0 auth_method 0 2024-03-13T07:34:54.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.074+0000 7f8556d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.074+0000 7f8556d9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.074+0000 7f8556d9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.074+0000 7fcc96ffd640 10 monclient(hunting): handle_auth_done global_id 4294 payload 824 2024-03-13T07:34:54.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.074+0000 7fcc96ffd640 1 -- 172.21.15.132:0/3861919449 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc90029690 msgr2=0x7fcc900b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.074+0000 7fcc96ffd640 1 --2- 172.21.15.132:0/3861919449 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc90029690 0x7fcc900b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.074+0000 7fcc96ffd640 1 -- 172.21.15.132:0/3861919449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc90006fe0 msgr2=0x7fcc9002a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.074+0000 7fcc96ffd640 1 --2- 172.21.15.132:0/3861919449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc90006fe0 0x7fcc9002a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.074+0000 7fcc96ffd640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.074+0000 7fcc96ffd640 1 monclient: found mon.noname-b 2024-03-13T07:34:54.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.074+0000 7fcc96ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.074+0000 7fcc96ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.074+0000 7fcc96ffd640 1 -- 172.21.15.132:0/3861919449 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9002a910 con 0x7fcc90031630 2024-03-13T07:34:54.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.075+0000 7f8556d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.075+0000 7f8556d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.075+0000 7f8556d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.077+0000 7f8556d9d640 10 monclient(hunting): handle_auth_done global_id 4290 payload 274 2024-03-13T07:34:54.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.077+0000 7f8556d9d640 1 -- 172.21.15.132:0/1711219165 >> v1:172.21.15.195:6789/0 conn(0x7f855813f950 legacy=0x7f85580afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.077+0000 7f8556d9d640 1 -- 172.21.15.132:0/1711219165 >> v1:172.21.15.132:6789/0 conn(0x7f85580b0270 legacy=0x7f855814cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.077+0000 7f8556d9d640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.077+0000 7f8556d9d640 1 monclient: found mon.noname-a 2024-03-13T07:34:54.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.077+0000 7f8556d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.077+0000 7f8556d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.077+0000 7f8556d9d640 1 -- 172.21.15.132:0/1711219165 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85580b17e0 con 0x7f85580b53e0 2024-03-13T07:34:54.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.077+0000 7f4ff27a2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fec133fb0 0x7f4fec134460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.078+0000 7f4ff27a2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fec133fb0 0x7f4fec134460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:48840/0 (socket says 172.21.15.132:48840) 2024-03-13T07:34:54.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.078+0000 7f3ce5254640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ce01538d0 0x7f3ce006fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.078+0000 7f4ff27a2640 1 -- 172.21.15.132:0/2509614961 learned_addr learned my addr 172.21.15.132:0/2509614961 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.078+0000 7f4ff27a2640 10 monclient(hunting): get_auth_request con 0x7f4fec133fb0 auth_method 0 2024-03-13T07:34:54.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.078+0000 7f4ff27a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.078+0000 7f4ff27a2640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.131 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.078+0000 7f4ff27a2640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.078+0000 7f3ce5254640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ce01538d0 0x7f3ce006fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:48856/0 (socket says 172.21.15.132:48856) 2024-03-13T07:34:54.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.078+0000 7f3ce5254640 1 -- 172.21.15.132:0/733602661 learned_addr learned my addr 172.21.15.132:0/733602661 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.078+0000 7f3ce5254640 10 monclient(hunting): get_auth_request con 0x7f3ce01538d0 auth_method 0 2024-03-13T07:34:54.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.078+0000 7f3ce5254640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.078+0000 7f3ce5254640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.078+0000 7f3ce5254640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.078+0000 7f3cdffff640 1 --2- 172.21.15.132:0/733602661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce0152de0 0x7f3ce006f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.079+0000 7f4ff27a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.079+0000 7f4ff27a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.079+0000 7f4ff27a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.079+0000 7fcc96ffd640 1 --2- 172.21.15.132:0/3861919449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc90031630 0x7fcc900c1a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400d5d0 tx=0x7fcc8400da90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.079+0000 7fcc83fff640 10 client.?.objecter ms_handle_connect 0x7fcc90031630 2024-03-13T07:34:54.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.079+0000 7fcc83fff640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:54.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f3ce5254640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f3ce5254640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f3ce5254640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f5084946640 1 -- 172.21.15.132:0/3933159473 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50780363f0 con 0x7f508006ec00 2024-03-13T07:34:54.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f5084946640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f5084946640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:54.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f5084946640 10 monclient: dump: 2024-03-13T07:34:54.135 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.135 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.135 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.135 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.135 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.135 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.135 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.135 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.135 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.135 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f5084946640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f5084946640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f5084946640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f5084946640 1 -- 172.21.15.132:0/3933159473 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5078036590 con 0x7f508006ec00 2024-03-13T07:34:54.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f5084946640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f5084946640 1 -- 172.21.15.132:0/3933159473 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5078036960 con 0x7f508006ec00 2024-03-13T07:34:54.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f5084946640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f5084946640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:54.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f5084946640 10 monclient: dump: 2024-03-13T07:34:54.136 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.136 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.136 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.136 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.136 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.136 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.136 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.136 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.137 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.137 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f5087bac640 5 monclient: authenticate success, global_id 4274 2024-03-13T07:34:54.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f5087bac640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:54.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f5087bac640 4 set_mon_vals no callback set 2024-03-13T07:34:54.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f8556d9d640 1 --2- 172.21.15.132:0/1711219165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85580b53e0 0x7f85580b1c00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8548009d90 tx=0x7f854802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dced44c346c95e1e server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7fb88ffff640 1 --2- 172.21.15.132:0/2611773776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb888030a80 0x7fb880006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f4ff17a0640 1 --2- 172.21.15.132:0/2509614961 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec13bf60 0x7f4fec13f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f3cdffff640 10 monclient(hunting): get_auth_request con 0x7f3ce0152de0 auth_method 0 2024-03-13T07:34:54.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f3cdffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f3cdffff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.080+0000 7f3cdffff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7f4ff17a0640 10 monclient(hunting): get_auth_request con 0x7f4fec13bf60 auth_method 0 2024-03-13T07:34:54.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7f4ff17a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7f4ff17a0640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7f4ff17a0640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7f5087bac640 10 monclient: shutdown 2024-03-13T07:34:54.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7f5087bac640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7f5087bac640 1 -- 172.21.15.132:0/3933159473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508006ec00 msgr2=0x7f508006f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7f5087bac640 1 --2- 172.21.15.132:0/3933159473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508006ec00 0x7f508006f820 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5078009730 tx=0x7f507802e350 comp rx=0 tx=0).stop 2024-03-13T07:34:54.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7f5087bac640 1 -- 172.21.15.132:0/3933159473 shutdown_connections 2024-03-13T07:34:54.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7f5087bac640 1 --2- 172.21.15.132:0/3933159473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f508006ec00 0x7f508006f820 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7f5087bac640 1 -- 172.21.15.132:0/3933159473 >> 172.21.15.132:0/3933159473 conn(0x7f50800b3c40 msgr2=0x7f50800b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7fb88ffff640 10 monclient(hunting): get_auth_request con 0x7fb888030a80 auth_method 0 2024-03-13T07:34:54.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7fb88ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7fb88ffff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7fb88ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-03-13T07:34:54.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7f5087bac640 1 -- 172.21.15.132:0/3933159473 shutdown_connections 2024-03-13T07:34:54.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7f5087bac640 1 -- 172.21.15.132:0/3933159473 wait complete. 2024-03-13T07:34:54.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7fb88ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7fb88ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.081+0000 7fb88ffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-03-13T07:34:54.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f4ff17a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f4ff17a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f4ff17a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f5087bac640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f692e0f3640 10 monclient(hunting): handle_auth_done global_id 4284 payload 274 2024-03-13T07:34:54.143 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f5087bac640 10 monclient: monmap: 2024-03-13T07:34:54.143 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.143 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:54.143 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:54.143 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:54.143 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.143 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.143 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:54.143 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:54.143 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:54.143 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.144 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f692e0f3640 1 -- 172.21.15.132:0/2683855404 >> v1:172.21.15.195:6789/0 conn(0x7f6928072270 legacy=0x7f6928072650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.144 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f692e0f3640 1 -- 172.21.15.132:0/2683855404 >> v1:172.21.15.132:6789/0 conn(0x7f692806fc70 legacy=0x7f692813f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.144 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f692e0f3640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.144 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f692e0f3640 1 monclient: found mon.noname-a 2024-03-13T07:34:54.144 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f692e0f3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f692e0f3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f692e0f3640 1 -- 172.21.15.132:0/2683855404 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692813fac0 con 0x7f69280ae620 2024-03-13T07:34:54.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f4ff27a2640 10 monclient(hunting): handle_auth_done global_id 4297 payload 824 2024-03-13T07:34:54.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f4ff27a2640 1 -- 172.21.15.132:0/2509614961 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec13bf60 msgr2=0x7f4fec13f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f4ff27a2640 1 --2- 172.21.15.132:0/2509614961 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fec13bf60 0x7f4fec13f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f4ff27a2640 1 -- 172.21.15.132:0/2509614961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fec06e820 msgr2=0x7f4fec140040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f4ff27a2640 1 --2- 172.21.15.132:0/2509614961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fec06e820 0x7f4fec140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f5087bac640 1 librados: starting msgr at 2024-03-13T07:34:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f5087bac640 1 librados: starting objecter 2024-03-13T07:34:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f4ff27a2640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f4ff27a2640 1 monclient: found mon.noname-b 2024-03-13T07:34:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.083+0000 7f5087bac640 1 Processor -- start 2024-03-13T07:34:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f4ff27a2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f4ff27a2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.082+0000 7f4ff27a2640 1 -- 172.21.15.132:0/2509614961 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec0aa4a0 con 0x7f4fec133fb0 2024-03-13T07:34:54.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.083+0000 7f5087bac640 1 -- start start 2024-03-13T07:34:54.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.083+0000 7f5087bac640 1 librados: setting wanted keys 2024-03-13T07:34:54.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.083+0000 7f5087bac640 1 librados: calling monclient init 2024-03-13T07:34:54.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.083+0000 7f5087bac640 10 monclient: init 2024-03-13T07:34:54.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.083+0000 7fb8949b9640 1 --2- 172.21.15.132:0/2611773776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb880005bc0 0x7fb880005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.083+0000 7f5087bac640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.083+0000 7f5087bac640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T07:34:54.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.083+0000 7f5087bac640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f508006e820 0x7f5080136db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.084+0000 7f5087bac640 10 monclient(hunting): picked mon.noname-a con 0x7f508006e820 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.084+0000 7f5087bac640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50800717c0 0x7f50801372f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.084+0000 7f5087bac640 10 monclient(hunting): picked mon.noname-c con 0x7f50800717c0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.084+0000 7f5087bac640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50801db570 0x7f50801db9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.084+0000 7f5087bac640 10 monclient(hunting): picked mon.noname-b con 0x7f50801db570 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.084+0000 7f5087bac640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.084+0000 7f5087bac640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5080141c80 con 0x7f508006e820 2024-03-13T07:34:54.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.084+0000 7f5087bac640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.084+0000 7f5087bac640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5080141b00 con 0x7f50801db570 2024-03-13T07:34:54.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.084+0000 7f5087bac640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.084+0000 7f5087bac640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50800b4760 con 0x7f50800717c0 2024-03-13T07:34:54.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.084+0000 7f5087bac640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.084+0000 7f5087bac640 10 monclient(hunting): authenticate will time out at 1321.493172s 2024-03-13T07:34:54.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.087+0000 7f3ce5254640 10 monclient(hunting): handle_auth_done global_id 4300 payload 824 2024-03-13T07:34:54.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.087+0000 7f3ce5254640 1 -- 172.21.15.132:0/733602661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce0152de0 msgr2=0x7f3ce006f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.087+0000 7f3ce5254640 1 --2- 172.21.15.132:0/733602661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce0152de0 0x7f3ce006f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.087+0000 7f3ce5254640 1 -- 172.21.15.132:0/733602661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ce008b4a0 msgr2=0x7f3ce00b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.087+0000 7f3ce5254640 1 --2- 172.21.15.132:0/733602661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ce008b4a0 0x7f3ce00b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.087+0000 7f3ce5254640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.087+0000 7f3ce5254640 1 monclient: found mon.noname-b 2024-03-13T07:34:54.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.087+0000 7f3ce5254640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.087+0000 7f3ce5254640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.087+0000 7f3ce5254640 1 -- 172.21.15.132:0/733602661 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce0148ac0 con 0x7f3ce01538d0 2024-03-13T07:34:54.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.088+0000 7fb8949b9640 10 monclient(hunting): get_auth_request con 0x7fb880005bc0 auth_method 0 2024-03-13T07:34:54.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.088+0000 7fb8949b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.088+0000 7fb8949b9640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.088+0000 7fb8949b9640 10 monclient(hunting): _init_auth already have auth, reseting 2024-03-13T07:34:54.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb8949b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb8949b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb8949b9640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-03-13T07:34:54.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7f692e0f3640 1 --2- 172.21.15.132:0/2683855404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69280ae620 0x7f69280aea50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6928070860 tx=0x7f692402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11766216a0a1126 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb88ffff640 10 monclient(hunting): handle_auth_done global_id 4281 payload 293 2024-03-13T07:34:54.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb88ffff640 1 -- 172.21.15.132:0/2611773776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb880005bc0 msgr2=0x7fb880005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb88ffff640 1 --2- 172.21.15.132:0/2611773776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb880005bc0 0x7fb880005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb88ffff640 1 -- 172.21.15.132:0/2611773776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb880006f50 msgr2=0x7fb8800073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb88ffff640 1 --2- 172.21.15.132:0/2611773776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb880006f50 0x7fb8800073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb88ffff640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb88ffff640 1 monclient: found mon.c 2024-03-13T07:34:54.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb88ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb88ffff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb88ffff640 1 -- 172.21.15.132:0/2611773776 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb888034020 con 0x7fb888030a80 2024-03-13T07:34:54.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb88ffff640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb88ffff640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb88ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb8959bb640 5 monclient: authenticate success, global_id 4281 2024-03-13T07:34:54.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.091+0000 7fb8959bb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:54.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.092+0000 7f4ff27a2640 1 --2- 172.21.15.132:0/2509614961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fec133fb0 0x7f4fec134460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00c390 tx=0x7f4fdc00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.092+0000 7f4fe27fc640 10 client.?.objecter ms_handle_connect 0x7f4fec133fb0 2024-03-13T07:34:54.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.092+0000 7f4fe27fc640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:54.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.093+0000 7f3ce5254640 1 --2- 172.21.15.132:0/733602661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ce01538d0 0x7f3ce006fd80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009d90 tx=0x7f3cd8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.093+0000 7f3cdd7fa640 10 client.?.objecter ms_handle_connect 0x7f3ce01538d0 2024-03-13T07:34:54.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.093+0000 7f3cdd7fa640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:54.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.095+0000 7fb88ffff640 1 --2- 172.21.15.132:0/2611773776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb888030a80 0x7fb880006a10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00c260 tx=0x7fb880007a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d67591a6278b0f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855659c640 1 -- 172.21.15.132:0/1711219165 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548039070 con 0x7f85580b53e0 2024-03-13T07:34:54.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855659c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855659c640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:54.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855659c640 10 monclient: dump: 2024-03-13T07:34:54.156 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.156 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.156 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.156 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.156 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.156 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.156 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.156 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.156 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.156 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855659c640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855659c640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855659c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855659c640 1 -- 172.21.15.132:0/1711219165 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854802cbd0 con 0x7f85580b53e0 2024-03-13T07:34:54.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855d24b640 5 monclient: authenticate success, global_id 4290 2024-03-13T07:34:54.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855d24b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:54.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855659c640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855d24b640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:54.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855659c640 1 -- 172.21.15.132:0/1711219165 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548034440 con 0x7f85580b53e0 2024-03-13T07:34:54.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855d24b640 4 set_mon_vals no callback set 2024-03-13T07:34:54.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855659c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855659c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:54.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f855659c640 10 monclient: dump: 2024-03-13T07:34:54.157 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.157 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.157 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.157 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.157 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.157 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.158 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.158 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.158 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.158 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.100+0000 7f5085147640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50800717c0 0x7f50801372f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.101+0000 7f855d24b640 10 monclient: shutdown 2024-03-13T07:34:54.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.101+0000 7f855d24b640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.101+0000 7f855d24b640 1 -- 172.21.15.132:0/1711219165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85580b53e0 msgr2=0x7f85580b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.101+0000 7f855d24b640 1 --2- 172.21.15.132:0/1711219165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85580b53e0 0x7f85580b1c00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8548009d90 tx=0x7f854802be20 comp rx=0 tx=0).stop 2024-03-13T07:34:54.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.101+0000 7f855d24b640 1 -- 172.21.15.132:0/1711219165 shutdown_connections 2024-03-13T07:34:54.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.101+0000 7f855d24b640 1 --2- 172.21.15.132:0/1711219165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85580b53e0 0x7f85580b1c00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.101+0000 7f855d24b640 1 -- 172.21.15.132:0/1711219165 >> 172.21.15.132:0/1711219165 conn(0x7f855806d2b0 msgr2=0x7f855806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.101+0000 7f855d24b640 1 -- 172.21.15.132:0/1711219165 shutdown_connections 2024-03-13T07:34:54.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.101+0000 7f855d24b640 1 -- 172.21.15.132:0/1711219165 wait complete. 2024-03-13T07:34:54.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.101+0000 7f5085147640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50800717c0 0x7f50801372f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:42720/0 (socket says 172.21.15.132:42720) 2024-03-13T07:34:54.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.101+0000 7f5085147640 1 -- 172.21.15.132:0/3235253391 learned_addr learned my addr 172.21.15.132:0/3235253391 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.101+0000 7f5085147640 10 monclient(hunting): get_auth_request con 0x7f50800717c0 auth_method 0 2024-03-13T07:34:54.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.101+0000 7f5085147640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.101+0000 7f5085147640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.101+0000 7f5085147640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.102+0000 7fb88f7fe640 1 -- 172.21.15.132:0/2611773776 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c007d70 con 0x7fb888030a80 2024-03-13T07:34:54.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.102+0000 7fb88f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.102+0000 7fb88f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:54.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.102+0000 7fb88f7fe640 10 monclient: dump: 2024-03-13T07:34:54.161 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.161 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.161 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.161 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.161 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.161 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.161 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.161 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.162 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.162 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.102+0000 7fb88f7fe640 1 -- 172.21.15.132:0/2611773776 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87c030070 con 0x7fb888030a80 2024-03-13T07:34:54.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.102+0000 7fb88f7fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.102+0000 7fb88f7fe640 1 -- 172.21.15.132:0/2611773776 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c02cdd0 con 0x7fb888030a80 2024-03-13T07:34:54.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.102+0000 7fb88f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.102+0000 7fb88f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:54.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.102+0000 7fb88f7fe640 10 monclient: dump: 2024-03-13T07:34:54.162 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.162 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.162 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.162 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.162 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.162 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.163 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.163 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.163 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.163 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.102+0000 7fb8959bb640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:54.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.102+0000 7fb8959bb640 4 set_mon_vals no callback set 2024-03-13T07:34:54.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.102+0000 7f855d24b640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.102+0000 7f855d24b640 10 monclient: monmap: 2024-03-13T07:34:54.164 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.164 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:54.164 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:54.164 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:54.164 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.164 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.164 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:54.164 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:54.164 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:54.164 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7f855d24b640 1 librados: starting msgr at 2024-03-13T07:34:54.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7f855d24b640 1 librados: starting objecter 2024-03-13T07:34:54.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7fb8959bb640 10 monclient: shutdown 2024-03-13T07:34:54.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7fb8959bb640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7fb8959bb640 1 -- 172.21.15.132:0/2611773776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb888030a80 msgr2=0x7fb880006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7fb8959bb640 1 --2- 172.21.15.132:0/2611773776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb888030a80 0x7fb880006a10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00c260 tx=0x7fb880007a40 comp rx=0 tx=0).stop 2024-03-13T07:34:54.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7fb8951ba640 1 -- 172.21.15.132:0/2611773776 reap_dead start 2024-03-13T07:34:54.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7f855d24b640 1 Processor -- start 2024-03-13T07:34:54.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7fb8959bb640 1 -- 172.21.15.132:0/2611773776 shutdown_connections 2024-03-13T07:34:54.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7fb8959bb640 1 -- 172.21.15.132:0/2611773776 >> 172.21.15.132:0/2611773776 conn(0x7fb88803a740 msgr2=0x7fb888036b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7f5085147640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7f5085147640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7f5085147640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7f855d24b640 1 -- start start 2024-03-13T07:34:54.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7f855d24b640 1 librados: setting wanted keys 2024-03-13T07:34:54.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7fb8959bb640 1 -- 172.21.15.132:0/2611773776 shutdown_connections 2024-03-13T07:34:54.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7f855d24b640 1 librados: calling monclient init 2024-03-13T07:34:54.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7f855d24b640 10 monclient: init 2024-03-13T07:34:54.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.103+0000 7fb8959bb640 1 -- 172.21.15.132:0/2611773776 wait complete. 2024-03-13T07:34:54.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f855d24b640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f855d24b640 10 monclient: _add_conns ranks=[2,1,0] 2024-03-13T07:34:54.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4fe27fc640 1 -- 172.21.15.132:0/2509614961 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc017070 con 0x7f4fec133fb0 2024-03-13T07:34:54.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4fe27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4fe27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:54.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4fe27fc640 10 monclient: dump: 2024-03-13T07:34:54.168 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.168 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.168 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.168 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.168 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.169 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.169 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.169 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.169 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.169 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4fe27fc640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4fe27fc640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4fe27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4fe27fc640 1 -- 172.21.15.132:0/2509614961 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc0035f0 con 0x7f4fec133fb0 2024-03-13T07:34:54.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4fe27fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4fe27fc640 1 -- 172.21.15.132:0/2509614961 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0039e0 con 0x7f4fec133fb0 2024-03-13T07:34:54.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4fe27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4fe27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:54.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f855d24b640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85580b0270 0x7f8558142b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f855d24b640 10 monclient(hunting): picked mon.noname-c con 0x7f85580b0270 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f855d24b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85580b53e0 0x7f85581db8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4fe27fc640 10 monclient: dump: 2024-03-13T07:34:54.170 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.170 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.170 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.170 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.170 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.170 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.171 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.171 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.171 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.171 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.171 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4fe37fe640 4 set_mon_vals no callback set 2024-03-13T07:34:54.171 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7fb8959bb640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.171 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7fb8959bb640 10 monclient: monmap: 2024-03-13T07:34:54.171 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.171 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:54.171 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:54.171 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:54.172 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.172 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.172 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:54.172 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:54.172 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:54.172 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7fb8959bb640 1 librados: starting msgr at 2024-03-13T07:34:54.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7fb8959bb640 1 librados: starting objecter 2024-03-13T07:34:54.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7fb8959bb640 1 Processor -- start 2024-03-13T07:34:54.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7fb8959bb640 1 -- start start 2024-03-13T07:34:54.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7fb8959bb640 1 librados: setting wanted keys 2024-03-13T07:34:54.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7fb8959bb640 1 librados: calling monclient init 2024-03-13T07:34:54.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7fb8959bb640 10 monclient: init 2024-03-13T07:34:54.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f5086149640 1 --2- 172.21.15.132:0/3235253391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50801db570 0x7f50801db9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f5086149640 10 monclient(hunting): get_auth_request con 0x7f50801db570 auth_method 0 2024-03-13T07:34:54.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f5086149640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f5086149640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f5086149640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f855d24b640 10 monclient(hunting): picked mon.noname-b con 0x7f85580b53e0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f855d24b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f855813f950 0x7f85581439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f855d24b640 10 monclient(hunting): picked mon.noname-a con 0x7f855813f950 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f855d24b640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f855d24b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855813fdc0 con 0x7f855813f950 2024-03-13T07:34:54.174 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f855d24b640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.174 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f855d24b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85580b48d0 con 0x7f85580b53e0 2024-03-13T07:34:54.174 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f855d24b640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.174 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f855d24b640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855813ff40 con 0x7f85580b0270 2024-03-13T07:34:54.174 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f855d24b640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.174 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f855d24b640 10 monclient(hunting): authenticate will time out at 1321.513638s 2024-03-13T07:34:54.174 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7f3cdd7fa640 1 -- 172.21.15.132:0/733602661 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8044030 con 0x7f3ce01538d0 2024-03-13T07:34:54.174 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7f3cdd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.174 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7f3cdd7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:54.174 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7f3cdd7fa640 10 monclient: dump: 2024-03-13T07:34:54.174 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.175 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.175 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.175 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.175 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.175 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.175 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.175 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.175 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.175 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.175 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7f3cdd7fa640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.175 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7f3cdd7fa640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.175 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7f3cdd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.175 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7f3cdd7fa640 1 -- 172.21.15.132:0/733602661 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd8035e50 con 0x7f3ce01538d0 2024-03-13T07:34:54.175 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7f3ce6cb7640 5 monclient: authenticate success, global_id 4300 2024-03-13T07:34:54.175 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7f3ce6cb7640 10 monclient: _renew_subs 2024-03-13T07:34:54.176 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7f3ce6cb7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.176 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7f3ce6cb7640 1 -- 172.21.15.132:0/733602661 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce00710f0 con 0x7f3ce01538d0 2024-03-13T07:34:54.176 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7f3cdd7fa640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.176 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7f3ce6cb7640 10 client.4300.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.176 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.105+0000 7f3cdeffd640 4 set_mon_vals no callback set 2024-03-13T07:34:54.176 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f3cdd7fa640 1 -- 172.21.15.132:0/733602661 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8039040 con 0x7f3ce01538d0 2024-03-13T07:34:54.176 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f3cdd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.176 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f3cdd7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:54.176 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f3cdd7fa640 10 monclient: dump: 2024-03-13T07:34:54.176 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.176 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.176 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.176 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.176 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.176 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.177 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.177 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.177 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.177 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f3ce6cb7640 10 monclient: _renew_subs 2024-03-13T07:34:54.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f3ce6cb7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f3ce6cb7640 1 -- 172.21.15.132:0/733602661 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce00717f0 con 0x7f3ce01538d0 2024-03-13T07:34:54.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f3ce6cb7640 1 librados: init done 2024-03-13T07:34:54.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4ff4205640 5 monclient: authenticate success, global_id 4297 2024-03-13T07:34:54.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4ff4205640 10 monclient: _renew_subs 2024-03-13T07:34:54.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4ff4205640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4ff4205640 1 -- 172.21.15.132:0/2509614961 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fec1406d0 con 0x7f4fec133fb0 2024-03-13T07:34:54.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4ff4205640 10 client.4297.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4ff4205640 10 monclient: _renew_subs 2024-03-13T07:34:54.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4ff4205640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4ff4205640 1 -- 172.21.15.132:0/2509614961 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fec1349a0 con 0x7f4fec133fb0 2024-03-13T07:34:54.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.104+0000 7f4ff4205640 1 librados: init done 2024-03-13T07:34:54.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7fb8959bb640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7fb8959bb640 10 monclient: _add_conns ranks=[1,2,0] 2024-03-13T07:34:54.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7fb8959bb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb888030a80 0x7fb88802e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7fb8959bb640 10 monclient(hunting): picked mon.noname-b con 0x7fb888030a80 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7fb8959bb640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb88802ec20 0x7fb88802c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f8556d9d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85580b0270 0x7f8558142b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7fb8959bb640 10 monclient(hunting): picked mon.noname-c con 0x7fb88802ec20 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7fb8959bb640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb88802c780 0x7fb8881587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7fb8959bb640 10 monclient(hunting): picked mon.noname-a con 0x7fb88802c780 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7fb8959bb640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7fb8959bb640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880c24b0 con 0x7fb88802c780 2024-03-13T07:34:54.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7fb8959bb640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7fb8959bb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880c4a10 con 0x7fb888030a80 2024-03-13T07:34:54.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7fb8959bb640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7fb8959bb640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880c5470 con 0x7fb88802ec20 2024-03-13T07:34:54.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7fb8959bb640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f4fcffff640 10 client.4297.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7fb8959bb640 10 monclient(hunting): authenticate will time out at 1321.515644s 2024-03-13T07:34:54.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f4fcffff640 15 client.4297.objecter _session_command_op_assign -1 1 2024-03-13T07:34:54.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f4fcffff640 20 client.4297.objecter _get_session osd=-1 returning homeless 2024-03-13T07:34:54.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f4fcffff640 10 client.4297.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f4fcffff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f8556d9d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85580b0270 0x7f8558142b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:42726/0 (socket says 172.21.15.132:42726) 2024-03-13T07:34:54.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f4fcffff640 1 -- 172.21.15.132:0/2509614961 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4fec04f000 con 0x7f4fec133fb0 2024-03-13T07:34:54.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f8556d9d640 1 -- 172.21.15.132:0/4257405221 learned_addr learned my addr 172.21.15.132:0/4257405221 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f8556d9d640 10 monclient(hunting): get_auth_request con 0x7f85580b0270 auth_method 0 2024-03-13T07:34:54.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f8556d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f8556d9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.106+0000 7f8556d9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc83fff640 1 -- 172.21.15.132:0/3861919449 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84004240 con 0x7fcc90031630 2024-03-13T07:34:54.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc83fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc83fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:54.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc83fff640 10 monclient: dump: 2024-03-13T07:34:54.183 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.183 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.183 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.183 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.184 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.184 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.184 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.184 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.184 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.184 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc83fff640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc83fff640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc83fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc83fff640 1 -- 172.21.15.132:0/3861919449 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc840043e0 con 0x7fcc90031630 2024-03-13T07:34:54.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc83fff640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc83fff640 1 -- 172.21.15.132:0/3861919449 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84003830 con 0x7fcc90031630 2024-03-13T07:34:54.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc83fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc83fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:54.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc83fff640 10 monclient: dump: 2024-03-13T07:34:54.185 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.185 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.185 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.185 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.185 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.185 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.185 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.185 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.185 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.185 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc9c9a3640 5 monclient: authenticate success, global_id 4294 2024-03-13T07:34:54.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc9c9a3640 10 monclient: _renew_subs 2024-03-13T07:34:54.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc9c9a3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc9c9a3640 1 -- 172.21.15.132:0/3861919449 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9002b730 con 0x7fcc90031630 2024-03-13T07:34:54.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7f854ffff640 1 --2- 172.21.15.132:0/4257405221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85580b53e0 0x7f85581db8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc9c9a3640 10 client.4294.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc9c9a3640 10 monclient: _renew_subs 2024-03-13T07:34:54.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc95ffb640 4 set_mon_vals no callback set 2024-03-13T07:34:54.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc9c9a3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc9c9a3640 1 -- 172.21.15.132:0/3861919449 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc900b1580 con 0x7fcc90031630 2024-03-13T07:34:54.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7fcc9c9a3640 1 librados: init done 2024-03-13T07:34:54.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7f3ce6cb7640 10 client.4300.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7f3ce6cb7640 15 client.4300.objecter _session_command_op_assign -1 1 2024-03-13T07:34:54.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7f3ce6cb7640 20 client.4300.objecter _get_session osd=-1 returning homeless 2024-03-13T07:34:54.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7f3ce6cb7640 10 client.4300.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7f3ce6cb7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.107+0000 7f3ce6cb7640 1 -- 172.21.15.132:0/733602661 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3ce0072640 con 0x7f3ce01538d0 2024-03-13T07:34:54.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.108+0000 7fcc9c9a3640 10 client.4294.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.108+0000 7fcc9c9a3640 15 client.4294.objecter _session_command_op_assign -1 1 2024-03-13T07:34:54.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.108+0000 7fcc9c9a3640 20 client.4294.objecter _get_session osd=-1 returning homeless 2024-03-13T07:34:54.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.108+0000 7fcc9c9a3640 10 client.4294.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.108+0000 7fcc9c9a3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.108+0000 7fcc9c9a3640 1 -- 172.21.15.132:0/3861919449 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc90005570 con 0x7fcc90031630 2024-03-13T07:34:54.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.110+0000 7f5085147640 10 monclient(hunting): handle_auth_done global_id 4292 payload 824 2024-03-13T07:34:54.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.110+0000 7f5085147640 1 -- 172.21.15.132:0/3235253391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50801db570 msgr2=0x7f50801db9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.110+0000 7f5085147640 1 --2- 172.21.15.132:0/3235253391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50801db570 0x7f50801db9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.110+0000 7f5085147640 1 -- 172.21.15.132:0/3235253391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f508006e820 msgr2=0x7f5080136db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.110+0000 7f5085147640 1 --2- 172.21.15.132:0/3235253391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f508006e820 0x7f5080136db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.110+0000 7f5085147640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.110+0000 7f5085147640 1 monclient: found mon.noname-c 2024-03-13T07:34:54.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.110+0000 7f5085147640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.110+0000 7f5085147640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.110+0000 7f5085147640 1 -- 172.21.15.132:0/3235253391 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50801ca610 con 0x7f50800717c0 2024-03-13T07:34:54.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.111+0000 7f8556d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.111+0000 7f8556d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.111+0000 7f8556d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.112+0000 7fb88ffff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb88802ec20 0x7fb88802c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.192 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.115+0000 7f8556d9d640 10 monclient(hunting): handle_auth_done global_id 4295 payload 824 2024-03-13T07:34:54.192 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.115+0000 7f8556d9d640 1 -- 172.21.15.132:0/4257405221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85580b53e0 msgr2=0x7f85581db8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.192 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.115+0000 7f8556d9d640 1 --2- 172.21.15.132:0/4257405221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85580b53e0 0x7f85581db8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.192 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.116+0000 7f8556d9d640 1 -- 172.21.15.132:0/4257405221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f855813f950 msgr2=0x7f85581439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.192 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.116+0000 7f8556d9d640 1 --2- 172.21.15.132:0/4257405221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f855813f950 0x7f85581439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.192 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.116+0000 7f8556d9d640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.192 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.116+0000 7f8556d9d640 1 monclient: found mon.noname-c 2024-03-13T07:34:54.192 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.116+0000 7f8556d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.192 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.116+0000 7f8556d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.193 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.116+0000 7f8556d9d640 1 -- 172.21.15.132:0/4257405221 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855813a9e0 con 0x7f85580b0270 2024-03-13T07:34:54.193 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.116+0000 7fb88ffff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb88802ec20 0x7fb88802c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:42736/0 (socket says 172.21.15.132:42736) 2024-03-13T07:34:54.193 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.116+0000 7fb88ffff640 1 -- 172.21.15.132:0/2155661044 learned_addr learned my addr 172.21.15.132:0/2155661044 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.193 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.116+0000 7fb88ffff640 10 monclient(hunting): get_auth_request con 0x7fb88802ec20 auth_method 0 2024-03-13T07:34:54.193 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.116+0000 7fb88ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.193 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.116+0000 7fb88ffff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.193 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.116+0000 7fb88ffff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.193 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.117+0000 7f5085147640 1 --2- 172.21.15.132:0/3235253391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50800717c0 0x7f50801372f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5078038040 tx=0x7f5078004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.117+0000 7f50767fc640 10 client.?.objecter ms_handle_connect 0x7f50800717c0 2024-03-13T07:34:54.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.117+0000 7f50767fc640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:54.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.117+0000 7fb88ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.117+0000 7fb88ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.117+0000 7fb88ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.123+0000 7fb88ffff640 10 monclient(hunting): handle_auth_done global_id 4298 payload 824 2024-03-13T07:34:54.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.123+0000 7fb88ffff640 1 -- 172.21.15.132:0/2155661044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb888030a80 msgr2=0x7fb88802e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.123+0000 7fb88ffff640 1 --2- 172.21.15.132:0/2155661044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb888030a80 0x7fb88802e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.123+0000 7fb88ffff640 1 -- 172.21.15.132:0/2155661044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb88802c780 msgr2=0x7fb8881587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.123+0000 7fb88ffff640 1 --2- 172.21.15.132:0/2155661044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb88802c780 0x7fb8881587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.123+0000 7fb88ffff640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.123+0000 7fb88ffff640 1 monclient: found mon.noname-c 2024-03-13T07:34:54.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.123+0000 7fb88ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.123+0000 7fb88ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.123+0000 7fb88ffff640 1 -- 172.21.15.132:0/2155661044 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88802ab70 con 0x7fb88802ec20 2024-03-13T07:34:54.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.125+0000 7f8556d9d640 1 --2- 172.21.15.132:0/4257405221 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85580b0270 0x7f8558142b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8548037040 tx=0x7f8548004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.125+0000 7f854f7fe640 10 client.?.objecter ms_handle_connect 0x7f85580b0270 2024-03-13T07:34:54.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.125+0000 7f854f7fe640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:54.196 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f50767fc640 1 -- 172.21.15.132:0/3235253391 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5078009510 con 0x7f50800717c0 2024-03-13T07:34:54.196 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f50767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f50767fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:54.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f50767fc640 10 monclient: dump: 2024-03-13T07:34:54.197 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.197 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.197 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.197 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.197 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.197 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.197 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.197 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.197 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.197 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f50767fc640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f50767fc640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f50767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f50767fc640 1 -- 172.21.15.132:0/3235253391 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5078045070 con 0x7f50800717c0 2024-03-13T07:34:54.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f50767fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f50767fc640 1 -- 172.21.15.132:0/3235253391 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507800e670 con 0x7f50800717c0 2024-03-13T07:34:54.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f5087bac640 5 monclient: authenticate success, global_id 4292 2024-03-13T07:34:54.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f5087bac640 10 monclient: _renew_subs 2024-03-13T07:34:54.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f5087bac640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f5087bac640 1 -- 172.21.15.132:0/3235253391 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5080137830 con 0x7f50800717c0 2024-03-13T07:34:54.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f5077fff640 4 set_mon_vals no callback set 2024-03-13T07:34:54.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f50767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f50767fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:54.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f50767fc640 10 monclient: dump: 2024-03-13T07:34:54.198 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.198 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.198 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.199 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.199 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.199 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.199 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.199 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.199 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.199 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f5087bac640 10 client.4292.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f5087bac640 10 monclient: _renew_subs 2024-03-13T07:34:54.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f5087bac640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f5087bac640 1 -- 172.21.15.132:0/3235253391 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50801dc0f0 con 0x7f50800717c0 2024-03-13T07:34:54.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.131+0000 7f5087bac640 1 librados: init done 2024-03-13T07:34:54.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.132+0000 7f5087bac640 10 client.4292.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.132+0000 7f5087bac640 15 client.4292.objecter _session_command_op_assign -1 1 2024-03-13T07:34:54.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.132+0000 7f5087bac640 20 client.4292.objecter _get_session osd=-1 returning homeless 2024-03-13T07:34:54.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.132+0000 7f5087bac640 10 client.4292.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.132+0000 7f5087bac640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.132+0000 7f5087bac640 1 -- 172.21.15.132:0/3235253391 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f508004f000 con 0x7f50800717c0 2024-03-13T07:34:54.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.134+0000 7f4fe27fc640 1 -- 172.21.15.132:0/2509614961 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4fdc003b80 con 0x7f4fec133fb0 2024-03-13T07:34:54.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.134+0000 7f4fe27fc640 1 --2- 172.21.15.132:0/2509614961 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4fd006d560 0x7f4fd006fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.134+0000 7fb88ffff640 1 --2- 172.21.15.132:0/2155661044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb88802ec20 0x7fb88802c240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb87c02c5a0 tx=0x7fb87c000fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.134+0000 7fb88d7fa640 10 client.?.objecter ms_handle_connect 0x7fb88802ec20 2024-03-13T07:34:54.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.134+0000 7fb88d7fa640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:54.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.134+0000 7f4ff1fa1640 1 --2- 172.21.15.132:0/2509614961 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4fd006d560 0x7f4fd006fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.134+0000 7f4ff1fa1640 10 monclient: get_auth_request con 0x7f4fd006d560 auth_method 0 2024-03-13T07:34:54.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.134+0000 7f854f7fe640 1 -- 172.21.15.132:0/4257405221 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548044020 con 0x7f85580b0270 2024-03-13T07:34:54.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f854f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f854f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:54.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f854f7fe640 10 monclient: dump: 2024-03-13T07:34:54.202 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.202 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.202 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.202 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.202 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.202 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.202 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.202 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.202 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.202 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f854f7fe640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f854f7fe640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f854f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f854f7fe640 1 -- 172.21.15.132:0/4257405221 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854803f670 con 0x7f85580b0270 2024-03-13T07:34:54.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f854f7fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f8555d9b640 4 set_mon_vals no callback set 2024-03-13T07:34:54.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f4ff1fa1640 1 --2- 172.21.15.132:0/2509614961 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4fd006d560 0x7f4fd006fa20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400ab90 tx=0x7f4fe4009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f4fe27fc640 10 client.4297.objecter ms_handle_connect 0x7f4fd006d560 2024-03-13T07:34:54.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f855d24b640 5 monclient: authenticate success, global_id 4295 2024-03-13T07:34:54.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f854f7fe640 1 -- 172.21.15.132:0/4257405221 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548039040 con 0x7f85580b0270 2024-03-13T07:34:54.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f855d24b640 10 monclient: _renew_subs 2024-03-13T07:34:54.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f855d24b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f855d24b640 1 -- 172.21.15.132:0/4257405221 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855813acc0 con 0x7f85580b0270 2024-03-13T07:34:54.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f855d24b640 10 client.4295.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f854f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f854f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:54.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f854f7fe640 10 monclient: dump: 2024-03-13T07:34:54.205 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.205 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.205 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.205 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.205 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.205 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.205 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.205 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.205 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.205 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f855d24b640 10 monclient: _renew_subs 2024-03-13T07:34:54.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f855d24b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f855d24b640 1 -- 172.21.15.132:0/4257405221 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85581dbfd0 con 0x7f85580b0270 2024-03-13T07:34:54.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.135+0000 7f855d24b640 1 librados: init done 2024-03-13T07:34:54.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.136+0000 7f855d24b640 10 client.4295.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.136+0000 7f855d24b640 15 client.4295.objecter _session_command_op_assign -1 1 2024-03-13T07:34:54.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.136+0000 7f855d24b640 20 client.4295.objecter _get_session osd=-1 returning homeless 2024-03-13T07:34:54.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.136+0000 7f855d24b640 10 client.4295.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.136+0000 7f855d24b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.136+0000 7f855d24b640 1 -- 172.21.15.132:0/4257405221 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f855804f000 con 0x7f85580b0270 2024-03-13T07:34:54.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.138+0000 7f4fe27fc640 1 -- 172.21.15.132:0/2509614961 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4fdc0564d0 con 0x7f4fec133fb0 2024-03-13T07:34:54.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.138+0000 7f4fe27fc640 10 client.4297.objecter ms_dispatch 0x7f4fec000b90 osd_map(25..25 src has 1..25) v4 2024-03-13T07:34:54.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.138+0000 7f4fe27fc640 3 client.4297.objecter handle_osd_map got epochs [25,25] > 0 2024-03-13T07:34:54.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.138+0000 7f4fe27fc640 3 client.4297.objecter handle_osd_map decoding full epoch 25 2024-03-13T07:34:54.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.138+0000 7f4fe27fc640 10 client.4297.objecter checking command 1 2024-03-13T07:34:54.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.138+0000 7f4fe27fc640 1 --2- 172.21.15.132:0/2509614961 >> [v2:172.21.15.195:6800/4106887608,v1:172.21.15.195:6801/4106887608] conn(0x7f4fd00739a0 0x7f4fd0075dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.138+0000 7f4fe27fc640 20 client.4297.objecter _get_session s=0x7f4fd00732d0 osd=6 3 2024-03-13T07:34:54.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.138+0000 7f4fe27fc640 20 client.4297.objecter put_session s=0x7f4fd00732d0 osd=6 3 2024-03-13T07:34:54.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.139+0000 7f4fe27fc640 15 client.4297.objecter _session_command_op_remove -1 1 2024-03-13T07:34:54.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.139+0000 7f4fe27fc640 20 client.4297.objecter _get_session s=0x7f4fd00732d0 osd=6 3 2024-03-13T07:34:54.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.139+0000 7f4fe27fc640 20 client.4297.objecter get_session s=0x7f4fd00732d0 osd=6 3 2024-03-13T07:34:54.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.139+0000 7f4fe27fc640 15 client.4297.objecter _session_command_op_assign 6 1 2024-03-13T07:34:54.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.139+0000 7f4fe27fc640 20 client.4297.objecter put_session s=0x7f4fd00732d0 osd=6 4 2024-03-13T07:34:54.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.139+0000 7f4fe27fc640 10 client.4297.objecter _send_command 1 2024-03-13T07:34:54.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.139+0000 7f4fe27fc640 1 -- 172.21.15.132:0/2509614961 --> [v2:172.21.15.195:6800/4106887608,v1:172.21.15.195:6801/4106887608] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fd0076470 con 0x7f4fd00739a0 2024-03-13T07:34:54.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.139+0000 7f4fe27fc640 20 client.4297.objecter dump_active .. 0 homeless 2024-03-13T07:34:54.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.139+0000 7f4ff17a0640 1 --2- 172.21.15.132:0/2509614961 >> [v2:172.21.15.195:6800/4106887608,v1:172.21.15.195:6801/4106887608] conn(0x7f4fd00739a0 0x7f4fd0075dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.140+0000 7f4ff17a0640 10 monclient: get_auth_request con 0x7f4fd00739a0 auth_method 0 2024-03-13T07:34:54.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.143+0000 7f4fe27fc640 10 client.4297.objecter ms_handle_connect 0x7f4fd00739a0 2024-03-13T07:34:54.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.143+0000 7f4ff17a0640 1 --2- 172.21.15.132:0/2509614961 >> [v2:172.21.15.195:6800/4106887608,v1:172.21.15.195:6801/4106887608] conn(0x7f4fd00739a0 0x7f4fd0075dc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.152+0000 7f3cdd7fa640 1 -- 172.21.15.132:0/733602661 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3cd8047070 con 0x7f3ce01538d0 2024-03-13T07:34:54.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.152+0000 7f3cdd7fa640 1 --2- 172.21.15.132:0/733602661 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3ccc06d5b0 0x7f3ccc06fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.153+0000 7f3ce4a53640 1 --2- 172.21.15.132:0/733602661 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3ccc06d5b0 0x7f3ccc06fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.153+0000 7f3ce4a53640 10 monclient: get_auth_request con 0x7f3ccc06d5b0 auth_method 0 2024-03-13T07:34:54.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.153+0000 7f3ce4a53640 1 --2- 172.21.15.132:0/733602661 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3ccc06d5b0 0x7f3ccc06fa70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4006a90 tx=0x7f3cd4006a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.153+0000 7f3cdd7fa640 10 client.4300.objecter ms_handle_connect 0x7f3ccc06d5b0 2024-03-13T07:34:54.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdd7fa640 1 -- 172.21.15.132:0/733602661 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3cd8079c50 con 0x7f3ce01538d0 2024-03-13T07:34:54.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdd7fa640 10 client.4300.objecter ms_dispatch 0x7f3ce0000b90 osd_map(25..25 src has 1..25) v4 2024-03-13T07:34:54.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdd7fa640 3 client.4300.objecter handle_osd_map got epochs [25,25] > 0 2024-03-13T07:34:54.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdd7fa640 3 client.4300.objecter handle_osd_map decoding full epoch 25 2024-03-13T07:34:54.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdd7fa640 10 client.4300.objecter checking command 1 2024-03-13T07:34:54.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdd7fa640 1 --2- 172.21.15.132:0/733602661 >> [v2:172.21.15.195:6816/1953907706,v1:172.21.15.195:6817/1953907706] conn(0x7f3ccc0739f0 0x7f3ccc075e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdd7fa640 20 client.4300.objecter _get_session s=0x7f3ccc073320 osd=5 3 2024-03-13T07:34:54.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdd7fa640 20 client.4300.objecter put_session s=0x7f3ccc073320 osd=5 3 2024-03-13T07:34:54.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdd7fa640 15 client.4300.objecter _session_command_op_remove -1 1 2024-03-13T07:34:54.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdd7fa640 20 client.4300.objecter _get_session s=0x7f3ccc073320 osd=5 3 2024-03-13T07:34:54.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdd7fa640 20 client.4300.objecter get_session s=0x7f3ccc073320 osd=5 3 2024-03-13T07:34:54.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdd7fa640 15 client.4300.objecter _session_command_op_assign 5 1 2024-03-13T07:34:54.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdd7fa640 20 client.4300.objecter put_session s=0x7f3ccc073320 osd=5 4 2024-03-13T07:34:54.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdd7fa640 10 client.4300.objecter _send_command 1 2024-03-13T07:34:54.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdd7fa640 1 -- 172.21.15.132:0/733602661 --> [v2:172.21.15.195:6816/1953907706,v1:172.21.15.195:6817/1953907706] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3cd8036ec0 con 0x7f3ccc0739f0 2024-03-13T07:34:54.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdd7fa640 20 client.4300.objecter dump_active .. 0 homeless 2024-03-13T07:34:54.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdffff640 1 --2- 172.21.15.132:0/733602661 >> [v2:172.21.15.195:6816/1953907706,v1:172.21.15.195:6817/1953907706] conn(0x7f3ccc0739f0 0x7f3ccc075e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.157+0000 7f3cdffff640 10 monclient: get_auth_request con 0x7f3ccc0739f0 auth_method 0 2024-03-13T07:34:54.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.160+0000 7f3cdd7fa640 10 client.4300.objecter ms_handle_connect 0x7f3ccc0739f0 2024-03-13T07:34:54.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.160+0000 7f3cdffff640 1 --2- 172.21.15.132:0/733602661 >> [v2:172.21.15.195:6816/1953907706,v1:172.21.15.195:6817/1953907706] conn(0x7f3ccc0739f0 0x7f3ccc075e10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.166+0000 7f854f7fe640 1 -- 172.21.15.132:0/4257405221 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8548047020 con 0x7f85580b0270 2024-03-13T07:34:54.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.167+0000 7f854f7fe640 1 --2- 172.21.15.132:0/4257405221 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f853806d5b0 0x7f853806fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.167+0000 7f854ffff640 1 --2- 172.21.15.132:0/4257405221 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f853806d5b0 0x7f853806fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.167+0000 7f854ffff640 10 monclient: get_auth_request con 0x7f853806d5b0 auth_method 0 2024-03-13T07:34:54.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.168+0000 7f854f7fe640 10 client.4295.objecter ms_handle_connect 0x7f853806d5b0 2024-03-13T07:34:54.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.168+0000 7f854ffff640 1 --2- 172.21.15.132:0/4257405221 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f853806d5b0 0x7f853806fa70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8558143350 tx=0x7f8540009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07a67fc640 1 -- 172.21.15.132:0/1617044316 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a002c9f0 con 0x7f07a8149850 2024-03-13T07:34:54.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07a67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07a67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:54.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07a67fc640 10 monclient: dump: 2024-03-13T07:34:54.214 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.214 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.214 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.214 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.214 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.214 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.214 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.214 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.214 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.214 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07a67fc640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07a67fc640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07a67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07a67fc640 1 -- 172.21.15.132:0/1617044316 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a002cb90 con 0x7f07a8149850 2024-03-13T07:34:54.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07acba3640 5 monclient: authenticate success, global_id 4278 2024-03-13T07:34:54.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07acba3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:54.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07a67fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07acba3640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:54.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07acba3640 4 set_mon_vals no callback set 2024-03-13T07:34:54.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07a67fc640 1 -- 172.21.15.132:0/1617044316 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a00343f0 con 0x7f07a8149850 2024-03-13T07:34:54.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07a67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07a67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:54.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.172+0000 7f07a67fc640 10 monclient: dump: 2024-03-13T07:34:54.215 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.216 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.216 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.216 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.216 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.216 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.216 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.216 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.216 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.216 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f07acba3640 10 monclient: shutdown 2024-03-13T07:34:54.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f07acba3640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f07acba3640 1 -- 172.21.15.132:0/1617044316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07a8149850 msgr2=0x7f07a8151370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f07acba3640 1 --2- 172.21.15.132:0/1617044316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07a8149850 0x7f07a8151370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f07a806db60 tx=0x7f07a002be50 comp rx=0 tx=0).stop 2024-03-13T07:34:54.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f07acba3640 1 -- 172.21.15.132:0/1617044316 shutdown_connections 2024-03-13T07:34:54.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f07acba3640 1 --2- 172.21.15.132:0/1617044316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07a8149850 0x7f07a8151370 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.217 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f07acba3640 1 --2- 172.21.15.132:0/1617044316 >> v2:172.21.15.195:3300/0 conn(0x7f07a80ae380 0x7f07a8149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.217 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f07acba3640 1 -- 172.21.15.132:0/1617044316 >> 172.21.15.132:0/1617044316 conn(0x7f07a80b08a0 msgr2=0x7f07a80b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.217 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f07acba3640 1 -- 172.21.15.132:0/1617044316 shutdown_connections 2024-03-13T07:34:54.217 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f07acba3640 1 -- 172.21.15.132:0/1617044316 wait complete. 2024-03-13T07:34:54.217 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f854f7fe640 1 -- 172.21.15.132:0/4257405221 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f854807a000 con 0x7f85580b0270 2024-03-13T07:34:54.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f854f7fe640 10 client.4295.objecter ms_dispatch 0x7f8558000b90 osd_map(25..25 src has 1..25) v4 2024-03-13T07:34:54.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f854f7fe640 3 client.4295.objecter handle_osd_map got epochs [25,25] > 0 2024-03-13T07:34:54.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f854f7fe640 3 client.4295.objecter handle_osd_map decoding full epoch 25 2024-03-13T07:34:54.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f854f7fe640 10 client.4295.objecter checking command 1 2024-03-13T07:34:54.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f854f7fe640 1 --2- 172.21.15.132:0/4257405221 >> [v2:172.21.15.132:6810/1770057764,v1:172.21.15.132:6811/1770057764] conn(0x7f85380739f0 0x7f8538075e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f854f7fe640 20 client.4295.objecter _get_session s=0x7f8538073320 osd=3 3 2024-03-13T07:34:54.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f854f7fe640 20 client.4295.objecter put_session s=0x7f8538073320 osd=3 3 2024-03-13T07:34:54.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f854f7fe640 15 client.4295.objecter _session_command_op_remove -1 1 2024-03-13T07:34:54.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f854f7fe640 20 client.4295.objecter _get_session s=0x7f8538073320 osd=3 3 2024-03-13T07:34:54.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f854f7fe640 20 client.4295.objecter get_session s=0x7f8538073320 osd=3 3 2024-03-13T07:34:54.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f854f7fe640 15 client.4295.objecter _session_command_op_assign 3 1 2024-03-13T07:34:54.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f854f7fe640 20 client.4295.objecter put_session s=0x7f8538073320 osd=3 4 2024-03-13T07:34:54.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f854f7fe640 10 client.4295.objecter _send_command 1 2024-03-13T07:34:54.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f854f7fe640 1 -- 172.21.15.132:0/4257405221 --> [v2:172.21.15.132:6810/1770057764,v1:172.21.15.132:6811/1770057764] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f85380764c0 con 0x7f85380739f0 2024-03-13T07:34:54.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.173+0000 7f854f7fe640 20 client.4295.objecter dump_active .. 0 homeless 2024-03-13T07:34:54.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.174+0000 7f07acba3640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.174+0000 7f07acba3640 10 monclient: monmap: 2024-03-13T07:34:54.219 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.219 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:54.219 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:54.219 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:54.220 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.220 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.220 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:54.220 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:54.220 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:54.220 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.174+0000 7f07acba3640 1 librados: starting msgr at 2024-03-13T07:34:54.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.174+0000 7f07acba3640 1 librados: starting objecter 2024-03-13T07:34:54.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.174+0000 7f07acba3640 1 Processor -- start 2024-03-13T07:34:54.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.174+0000 7f855759e640 1 --2- 172.21.15.132:0/4257405221 >> [v2:172.21.15.132:6810/1770057764,v1:172.21.15.132:6811/1770057764] conn(0x7f85380739f0 0x7f8538075e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.174+0000 7f07acba3640 1 -- start start 2024-03-13T07:34:54.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.174+0000 7f07acba3640 1 librados: setting wanted keys 2024-03-13T07:34:54.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.174+0000 7f07acba3640 1 librados: calling monclient init 2024-03-13T07:34:54.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.174+0000 7f07acba3640 10 monclient: init 2024-03-13T07:34:54.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.174+0000 7f855759e640 10 monclient: get_auth_request con 0x7f85380739f0 auth_method 0 2024-03-13T07:34:54.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.175+0000 7f07acba3640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.175+0000 7f07acba3640 10 monclient: _add_conns ranks=[2,0,1] 2024-03-13T07:34:54.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.175+0000 7f07acba3640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07a80a9980 0x7f07a8128240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.175+0000 7f07acba3640 10 monclient(hunting): picked mon.noname-c con 0x7f07a80a9980 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.175+0000 7f07acba3640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07a80ae380 0x7f07a8145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.175+0000 7f07acba3640 10 monclient(hunting): picked mon.noname-a con 0x7f07a80ae380 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.175+0000 7f07acba3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07a8149850 0x7f07a8145950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.175+0000 7f07acba3640 10 monclient(hunting): picked mon.noname-b con 0x7f07a8149850 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.175+0000 7f07acba3640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.175+0000 7f07acba3640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a80b0180 con 0x7f07a80ae380 2024-03-13T07:34:54.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.175+0000 7f07acba3640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.175+0000 7f07acba3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a80aae80 con 0x7f07a8149850 2024-03-13T07:34:54.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.175+0000 7f07acba3640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.175+0000 7f07acba3640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a8149ca0 con 0x7f07a80a9980 2024-03-13T07:34:54.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.175+0000 7f07acba3640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.175+0000 7f07acba3640 10 monclient(hunting): authenticate will time out at 1321.584588s 2024-03-13T07:34:54.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.177+0000 7f854f7fe640 10 client.4295.objecter ms_handle_connect 0x7f85380739f0 2024-03-13T07:34:54.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.177+0000 7f855759e640 1 --2- 172.21.15.132:0/4257405221 >> [v2:172.21.15.132:6810/1770057764,v1:172.21.15.132:6811/1770057764] conn(0x7f85380739f0 0x7f8538075e10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.178+0000 7f692c8f0640 1 -- 172.21.15.132:0/2683855404 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924034070 con 0x7f69280ae620 2024-03-13T07:34:54.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.178+0000 7f692c8f0640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.178+0000 7f692c8f0640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:54.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.178+0000 7f692c8f0640 10 monclient: dump: 2024-03-13T07:34:54.226 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.226 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.226 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.226 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.226 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.226 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.226 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.226 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.226 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.227 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.178+0000 7f692c8f0640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.178+0000 7f692c8f0640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.178+0000 7f692c8f0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.178+0000 7f692c8f0640 1 -- 172.21.15.132:0/2683855404 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6924030840 con 0x7f69280ae620 2024-03-13T07:34:54.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.178+0000 7f692e8f4640 5 monclient: authenticate success, global_id 4284 2024-03-13T07:34:54.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.178+0000 7f692e8f4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:54.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.178+0000 7f692c8f0640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.178+0000 7f692c8f0640 1 -- 172.21.15.132:0/2683855404 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924030c10 con 0x7f69280ae620 2024-03-13T07:34:54.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.178+0000 7f692e8f4640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:54.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.178+0000 7f692e8f4640 4 set_mon_vals no callback set 2024-03-13T07:34:54.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.179+0000 7f692c8f0640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.179+0000 7f692c8f0640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:54.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.179+0000 7f692c8f0640 10 monclient: dump: 2024-03-13T07:34:54.227 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.228 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.228 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.228 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.228 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.228 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.228 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.228 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.228 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.228 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.179+0000 7f692e8f4640 10 monclient: shutdown 2024-03-13T07:34:54.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.179+0000 7f692e8f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.180+0000 7f692e8f4640 1 -- 172.21.15.132:0/2683855404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69280ae620 msgr2=0x7f69280aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.180+0000 7f692e8f4640 1 --2- 172.21.15.132:0/2683855404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69280ae620 0x7f69280aea50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6928070860 tx=0x7f692402b980 comp rx=0 tx=0).stop 2024-03-13T07:34:54.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.180+0000 7f692e8f4640 1 -- 172.21.15.132:0/2683855404 shutdown_connections 2024-03-13T07:34:54.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.180+0000 7f692e8f4640 1 --2- 172.21.15.132:0/2683855404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69280ae620 0x7f69280aea50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.180+0000 7f692e8f4640 1 -- 172.21.15.132:0/2683855404 >> 172.21.15.132:0/2683855404 conn(0x7f69280b4d60 msgr2=0x7f69280b5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.180+0000 7f692e8f4640 1 -- 172.21.15.132:0/2683855404 shutdown_connections 2024-03-13T07:34:54.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.180+0000 7f692e8f4640 1 -- 172.21.15.132:0/2683855404 wait complete. 2024-03-13T07:34:54.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.180+0000 7f07a6ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07a80ae380 0x7f07a8145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.181+0000 7f07a6ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07a80ae380 0x7f07a8145410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56646/0 (socket says 172.21.15.132:56646) 2024-03-13T07:34:54.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.181+0000 7f07a6ffd640 1 -- 172.21.15.132:0/3996886250 learned_addr learned my addr 172.21.15.132:0/3996886250 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.181+0000 7f07a6ffd640 10 monclient(hunting): get_auth_request con 0x7f07a80ae380 auth_method 0 2024-03-13T07:34:54.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.181+0000 7f07a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.181+0000 7f07a6ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.181+0000 7f07a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.181+0000 7f692e8f4640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.181+0000 7f692e8f4640 10 monclient: monmap: 2024-03-13T07:34:54.231 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.231 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:54.231 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:54.231 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:54.231 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.231 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.231 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:54.231 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:54.231 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:54.231 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.181+0000 7f692e8f4640 1 librados: starting msgr at 2024-03-13T07:34:54.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.181+0000 7f692e8f4640 1 librados: starting objecter 2024-03-13T07:34:54.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 1 Processor -- start 2024-03-13T07:34:54.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 1 -- start start 2024-03-13T07:34:54.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 1 librados: setting wanted keys 2024-03-13T07:34:54.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 1 librados: calling monclient init 2024-03-13T07:34:54.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 10 monclient: init 2024-03-13T07:34:54.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f07a6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f07a6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f07a6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 10 monclient: _add_conns ranks=[1,0,2] 2024-03-13T07:34:54.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f692806fc70 0x7f692813c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 10 monclient(hunting): picked mon.noname-b con 0x7f692806fc70 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6928072270 0x7f692813baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 10 monclient(hunting): picked mon.noname-a con 0x7f6928072270 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69280ae620 0x7f6928134910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 10 monclient(hunting): picked mon.noname-c con 0x7f69280ae620 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692814ad70 con 0x7f6928072270 2024-03-13T07:34:54.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692814abf0 con 0x7f692806fc70 2024-03-13T07:34:54.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692814aef0 con 0x7f69280ae620 2024-03-13T07:34:54.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.182+0000 7f692e8f4640 10 monclient(hunting): authenticate will time out at 1321.591872s 2024-03-13T07:34:54.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.183+0000 7fcc83fff640 1 -- 172.21.15.132:0/3861919449 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fcc840039d0 con 0x7fcc90031630 2024-03-13T07:34:54.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.184+0000 7fcc83fff640 1 --2- 172.21.15.132:0/3861919449 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcc7c06d5b0 0x7fcc7c06fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.184+0000 7fcc977fe640 1 --2- 172.21.15.132:0/3861919449 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcc7c06d5b0 0x7fcc7c06fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.184+0000 7fcc977fe640 10 monclient: get_auth_request con 0x7fcc7c06d5b0 auth_method 0 2024-03-13T07:34:54.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.184+0000 7fcc83fff640 10 client.4294.objecter ms_handle_connect 0x7fcc7c06d5b0 2024-03-13T07:34:54.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.184+0000 7fcc977fe640 1 --2- 172.21.15.132:0/3861919449 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcc7c06d5b0 0x7fcc7c06fa70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c036000 tx=0x7fcc8c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.185+0000 7f07a6ffd640 10 monclient(hunting): handle_auth_done global_id 4293 payload 824 2024-03-13T07:34:54.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.185+0000 7f07a6ffd640 1 -- 172.21.15.132:0/3996886250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07a80a9980 msgr2=0x7f07a8128240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.185+0000 7f07a6ffd640 1 --2- 172.21.15.132:0/3996886250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07a80a9980 0x7f07a8128240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.185+0000 7f07a6ffd640 1 -- 172.21.15.132:0/3996886250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07a8149850 msgr2=0x7f07a8145950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.185+0000 7f07a6ffd640 1 --2- 172.21.15.132:0/3996886250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07a8149850 0x7f07a8145950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.185+0000 7f07a6ffd640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.185+0000 7f07a6ffd640 1 monclient: found mon.noname-a 2024-03-13T07:34:54.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.185+0000 7f07a6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.185+0000 7f07a6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.185+0000 7f07a6ffd640 1 -- 172.21.15.132:0/3996886250 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a8129780 con 0x7f07a80ae380 2024-03-13T07:34:54.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.186+0000 7fcc83fff640 1 -- 172.21.15.132:0/3861919449 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcc84057c00 con 0x7fcc90031630 2024-03-13T07:34:54.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.186+0000 7fcc83fff640 10 client.4294.objecter ms_dispatch 0x7fcc98000b90 osd_map(25..25 src has 1..25) v4 2024-03-13T07:34:54.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.186+0000 7fcc83fff640 3 client.4294.objecter handle_osd_map got epochs [25,25] > 0 2024-03-13T07:34:54.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.186+0000 7fcc83fff640 3 client.4294.objecter handle_osd_map decoding full epoch 25 2024-03-13T07:34:54.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.186+0000 7fcc83fff640 10 client.4294.objecter checking command 1 2024-03-13T07:34:54.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.186+0000 7fcc83fff640 1 --2- 172.21.15.132:0/3861919449 >> [v2:172.21.15.195:6802/699638467,v1:172.21.15.195:6803/699638467] conn(0x7fcc7c0739f0 0x7fcc7c075e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.186+0000 7fcc83fff640 20 client.4294.objecter _get_session s=0x7fcc7c073320 osd=4 3 2024-03-13T07:34:54.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.186+0000 7fcc83fff640 20 client.4294.objecter put_session s=0x7fcc7c073320 osd=4 3 2024-03-13T07:34:54.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.186+0000 7fcc83fff640 15 client.4294.objecter _session_command_op_remove -1 1 2024-03-13T07:34:54.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.186+0000 7fcc83fff640 20 client.4294.objecter _get_session s=0x7fcc7c073320 osd=4 3 2024-03-13T07:34:54.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.186+0000 7fcc83fff640 20 client.4294.objecter get_session s=0x7fcc7c073320 osd=4 3 2024-03-13T07:34:54.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.186+0000 7fcc83fff640 15 client.4294.objecter _session_command_op_assign 4 1 2024-03-13T07:34:54.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.186+0000 7fcc83fff640 20 client.4294.objecter put_session s=0x7fcc7c073320 osd=4 4 2024-03-13T07:34:54.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.186+0000 7fcc83fff640 10 client.4294.objecter _send_command 1 2024-03-13T07:34:54.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.186+0000 7fcc83fff640 1 -- 172.21.15.132:0/3861919449 --> [v2:172.21.15.195:6802/699638467,v1:172.21.15.195:6803/699638467] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc8400ee50 con 0x7fcc7c0739f0 2024-03-13T07:34:54.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.186+0000 7fcc83fff640 20 client.4294.objecter dump_active .. 0 homeless 2024-03-13T07:34:54.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.187+0000 7fcc9dc05640 1 --2- 172.21.15.132:0/3861919449 >> [v2:172.21.15.195:6802/699638467,v1:172.21.15.195:6803/699638467] conn(0x7fcc7c0739f0 0x7fcc7c075e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.187+0000 7fcc9dc05640 10 monclient: get_auth_request con 0x7fcc7c0739f0 auth_method 0 2024-03-13T07:34:54.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.188+0000 7f50767fc640 1 -- 172.21.15.132:0/3235253391 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5078040420 con 0x7f50800717c0 2024-03-13T07:34:54.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.189+0000 7f07a6ffd640 1 --2- 172.21.15.132:0/3996886250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07a80ae380 0x7f07a8145410 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f079c00d800 tx=0x7f079c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.189+0000 7f078ffff640 10 client.?.objecter ms_handle_connect 0x7f07a80ae380 2024-03-13T07:34:54.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.189+0000 7f078ffff640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:54.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.189+0000 7f50767fc640 1 --2- 172.21.15.132:0/3235253391 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f506406d5e0 0x7f506406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.189+0000 7f5085948640 1 --2- 172.21.15.132:0/3235253391 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f506406d5e0 0x7f506406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.189+0000 7f5085948640 10 monclient: get_auth_request con 0x7f506406d5e0 auth_method 0 2024-03-13T07:34:54.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.189+0000 7f5085948640 1 --2- 172.21.15.132:0/3235253391 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f506406d5e0 0x7f506406faa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5070001560 tx=0x7f5070007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.189+0000 7f50767fc640 10 client.4292.objecter ms_handle_connect 0x7f506406d5e0 2024-03-13T07:34:54.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.189+0000 7f692d0f1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6928072270 0x7f692813baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.190+0000 7f692d0f1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6928072270 0x7f692813baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56652/0 (socket says 172.21.15.132:56652) 2024-03-13T07:34:54.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.190+0000 7f692d0f1640 1 -- 172.21.15.132:0/1569100031 learned_addr learned my addr 172.21.15.132:0/1569100031 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.190+0000 7f692d0f1640 10 monclient(hunting): get_auth_request con 0x7f6928072270 auth_method 0 2024-03-13T07:34:54.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.190+0000 7f692d0f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.190+0000 7f692d0f1640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.190+0000 7f692d0f1640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.191+0000 7fcc9dc05640 1 --2- 172.21.15.132:0/3861919449 >> [v2:172.21.15.195:6802/699638467,v1:172.21.15.195:6803/699638467] conn(0x7fcc7c0739f0 0x7fcc7c075e10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.191+0000 7fcc83fff640 10 client.4294.objecter ms_handle_connect 0x7fcc7c0739f0 2024-03-13T07:34:54.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.193+0000 7f692d0f1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.193+0000 7f692d0f1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.193+0000 7f692d0f1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.194+0000 7f078ffff640 1 -- 172.21.15.132:0/3996886250 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c00b640 con 0x7f07a80ae380 2024-03-13T07:34:54.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.194+0000 7f078ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.194+0000 7f078ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:54.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.194+0000 7f078ffff640 10 monclient: dump: 2024-03-13T07:34:54.242 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.242 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.242 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.242 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.242 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.242 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.242 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.243 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.243 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.243 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.194+0000 7f078ffff640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.194+0000 7f078ffff640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.194+0000 7f078ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.194+0000 7f078ffff640 1 -- 172.21.15.132:0/3996886250 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c019070 con 0x7f07a80ae380 2024-03-13T07:34:54.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.194+0000 7f078ffff640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.194+0000 7f078ffff640 1 -- 172.21.15.132:0/3996886250 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c00ba00 con 0x7f07a80ae380 2024-03-13T07:34:54.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.194+0000 7f078ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.194+0000 7f078ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:54.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.194+0000 7f078ffff640 10 monclient: dump: 2024-03-13T07:34:54.243 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.243 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.243 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.244 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.244 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.244 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.244 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.244 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.244 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.244 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.194+0000 7f07a5ffb640 4 set_mon_vals no callback set 2024-03-13T07:34:54.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f07acba3640 5 monclient: authenticate success, global_id 4293 2024-03-13T07:34:54.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f07acba3640 10 monclient: _renew_subs 2024-03-13T07:34:54.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f07acba3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f07acba3640 1 -- 172.21.15.132:0/3996886250 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a8145e90 con 0x7f07a80ae380 2024-03-13T07:34:54.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f07acba3640 10 client.4293.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f07acba3640 10 monclient: _renew_subs 2024-03-13T07:34:54.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f07acba3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f07acba3640 1 -- 172.21.15.132:0/3996886250 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a8128780 con 0x7f07a80ae380 2024-03-13T07:34:54.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f07acba3640 1 librados: init done 2024-03-13T07:34:54.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f07acba3640 10 client.4293.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f07acba3640 15 client.4293.objecter _session_command_op_assign -1 1 2024-03-13T07:34:54.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f50767fc640 1 -- 172.21.15.132:0/3235253391 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5078079f50 con 0x7f50800717c0 2024-03-13T07:34:54.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f50767fc640 10 client.4292.objecter ms_dispatch 0x7f5080000b90 osd_map(25..25 src has 1..25) v4 2024-03-13T07:34:54.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f50767fc640 3 client.4292.objecter handle_osd_map got epochs [25,25] > 0 2024-03-13T07:34:54.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f50767fc640 3 client.4292.objecter handle_osd_map decoding full epoch 25 2024-03-13T07:34:54.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f07acba3640 20 client.4293.objecter _get_session osd=-1 returning homeless 2024-03-13T07:34:54.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f07acba3640 10 client.4293.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f07acba3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f07acba3640 1 -- 172.21.15.132:0/3996886250 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f07a806a5c0 con 0x7f07a80ae380 2024-03-13T07:34:54.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f50767fc640 10 client.4292.objecter checking command 1 2024-03-13T07:34:54.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f50767fc640 1 --2- 172.21.15.132:0/3235253391 >> [v2:172.21.15.132:6826/958254560,v1:172.21.15.132:6827/958254560] conn(0x7f5064073a20 0x7f5064075e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f50767fc640 20 client.4292.objecter _get_session s=0x7f5064073350 osd=2 3 2024-03-13T07:34:54.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f50767fc640 20 client.4292.objecter put_session s=0x7f5064073350 osd=2 3 2024-03-13T07:34:54.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f50767fc640 15 client.4292.objecter _session_command_op_remove -1 1 2024-03-13T07:34:54.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f50767fc640 20 client.4292.objecter _get_session s=0x7f5064073350 osd=2 3 2024-03-13T07:34:54.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f50767fc640 20 client.4292.objecter get_session s=0x7f5064073350 osd=2 3 2024-03-13T07:34:54.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f50767fc640 15 client.4292.objecter _session_command_op_assign 2 1 2024-03-13T07:34:54.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f50767fc640 20 client.4292.objecter put_session s=0x7f5064073350 osd=2 4 2024-03-13T07:34:54.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f50767fc640 10 client.4292.objecter _send_command 1 2024-03-13T07:34:54.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f50767fc640 1 -- 172.21.15.132:0/3235253391 --> [v2:172.21.15.132:6826/958254560,v1:172.21.15.132:6827/958254560] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5078045e70 con 0x7f5064073a20 2024-03-13T07:34:54.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.195+0000 7f50767fc640 20 client.4292.objecter dump_active .. 0 homeless 2024-03-13T07:34:54.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.196+0000 7f4fe27fc640 1 -- 172.21.15.132:0/2509614961 <== osd.6 v2:172.21.15.195:6800/4106887608 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4fd0076470 con 0x7f4fd00739a0 2024-03-13T07:34:54.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.196+0000 7f4fe27fc640 10 client.4297.objecter ms_dispatch 0x7f4fec000b90 command_reply(tid 1: 0 ) v1 2024-03-13T07:34:54.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.196+0000 7f4fe27fc640 10 client.4297.objecter _finish_command 1 = system:0 2024-03-13T07:34:54.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.196+0000 7f4fe27fc640 20 client.4297.objecter put_session s=0x7f4fd00732d0 osd=6 4 2024-03-13T07:34:54.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.196+0000 7f4fe27fc640 15 client.4297.objecter _session_command_op_remove 6 1 2024-03-13T07:34:54.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.197+0000 7f5086149640 1 --2- 172.21.15.132:0/3235253391 >> [v2:172.21.15.132:6826/958254560,v1:172.21.15.132:6827/958254560] conn(0x7f5064073a20 0x7f5064075e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.197+0000 7f5086149640 10 monclient: get_auth_request con 0x7f5064073a20 auth_method 0 2024-03-13T07:34:54.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.197+0000 7f692d0f1640 10 monclient(hunting): handle_auth_done global_id 4296 payload 824 2024-03-13T07:34:54.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7f692d0f1640 1 -- 172.21.15.132:0/1569100031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69280ae620 msgr2=0x7f6928134910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7f692d0f1640 1 --2- 172.21.15.132:0/1569100031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69280ae620 0x7f6928134910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7f692d0f1640 1 -- 172.21.15.132:0/1569100031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f692806fc70 msgr2=0x7f692813c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7f692d0f1640 1 --2- 172.21.15.132:0/1569100031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f692806fc70 0x7f692813c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7f692d0f1640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7f692d0f1640 1 monclient: found mon.noname-a 2024-03-13T07:34:54.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7f692d0f1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7f692d0f1640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7f692d0f1640 1 -- 172.21.15.132:0/1569100031 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69280a9ac0 con 0x7f6928072270 2024-03-13T07:34:54.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb88d7fa640 1 -- 172.21.15.132:0/2155661044 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c0049f0 con 0x7fb88802ec20 2024-03-13T07:34:54.251 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb88d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.251 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb88d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:54.251 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb88d7fa640 10 monclient: dump: 2024-03-13T07:34:54.251 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.251 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.251 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.251 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.251 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.251 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.251 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.251 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.251 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.251 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.251 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb88d7fa640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb88d7fa640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb88d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb88d7fa640 1 -- 172.21.15.132:0/2155661044 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87c004b90 con 0x7fb88802ec20 2024-03-13T07:34:54.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb88d7fa640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb88effd640 4 set_mon_vals no callback set 2024-03-13T07:34:54.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb88d7fa640 1 -- 172.21.15.132:0/2155661044 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c007070 con 0x7fb88802ec20 2024-03-13T07:34:54.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb88d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb88d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:54.252 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb88d7fa640 10 monclient: dump: 2024-03-13T07:34:54.252 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.252 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.252 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.252 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.252 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.252 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.253 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.253 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.253 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.253 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.253 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb8959bb640 5 monclient: authenticate success, global_id 4298 2024-03-13T07:34:54.253 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb8959bb640 10 monclient: _renew_subs 2024-03-13T07:34:54.253 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb8959bb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.253 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb8959bb640 1 -- 172.21.15.132:0/2155661044 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88802f4a0 con 0x7fb88802ec20 2024-03-13T07:34:54.253 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb8959bb640 10 client.4298.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.253 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb8959bb640 10 monclient: _renew_subs 2024-03-13T07:34:54.253 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb8959bb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.253 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb8959bb640 1 -- 172.21.15.132:0/2155661044 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88802cc00 con 0x7fb88802ec20 2024-03-13T07:34:54.253 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.198+0000 7fb8959bb640 1 librados: init done 2024-03-13T07:34:54.253 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.200+0000 7fb8959bb640 10 client.4298.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.253 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.200+0000 7fb8959bb640 15 client.4298.objecter _session_command_op_assign -1 1 2024-03-13T07:34:54.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.200+0000 7fb8959bb640 20 client.4298.objecter _get_session osd=-1 returning homeless 2024-03-13T07:34:54.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.200+0000 7fb8959bb640 10 client.4298.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.200+0000 7fb8959bb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.200+0000 7fb8959bb640 1 -- 172.21.15.132:0/2155661044 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb8880c2010 con 0x7fb88802ec20 2024-03-13T07:34:54.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.202+0000 7f5086149640 1 --2- 172.21.15.132:0/3235253391 >> [v2:172.21.15.132:6826/958254560,v1:172.21.15.132:6827/958254560] conn(0x7f5064073a20 0x7f5064075e40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.202+0000 7f50767fc640 10 client.4292.objecter ms_handle_connect 0x7f5064073a20 2024-03-13T07:34:54.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.206+0000 7f3cdd7fa640 1 -- 172.21.15.132:0/733602661 <== osd.5 v2:172.21.15.195:6816/1953907706 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3cd8036ec0 con 0x7f3ccc0739f0 2024-03-13T07:34:54.255 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.206+0000 7f3cdd7fa640 10 client.4300.objecter ms_dispatch 0x7f3ce0000b90 command_reply(tid 1: 0 ) v1 2024-03-13T07:34:54.255 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.206+0000 7f3cdd7fa640 10 client.4300.objecter _finish_command 1 = system:0 2024-03-13T07:34:54.255 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.206+0000 7f3cdd7fa640 20 client.4300.objecter put_session s=0x7f3ccc073320 osd=5 4 2024-03-13T07:34:54.255 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.206+0000 7f3cdd7fa640 15 client.4300.objecter _session_command_op_remove 5 1 2024-03-13T07:34:54.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.224+0000 7f078ffff640 1 -- 172.21.15.132:0/3996886250 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f079c00bba0 con 0x7f07a80ae380 2024-03-13T07:34:54.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.224+0000 7f078ffff640 1 --2- 172.21.15.132:0/3996886250 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f079406d5e0 0x7f079406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.225+0000 7f07a77fe640 1 --2- 172.21.15.132:0/3996886250 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f079406d5e0 0x7f079406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.225+0000 7f07a77fe640 10 monclient: get_auth_request con 0x7f079406d5e0 auth_method 0 2024-03-13T07:34:54.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.225+0000 7f07a77fe640 1 --2- 172.21.15.132:0/3996886250 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f079406d5e0 0x7f079406faa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0798001560 tx=0x7f0798007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.225+0000 7f078ffff640 10 client.4293.objecter ms_handle_connect 0x7f079406d5e0 2024-03-13T07:34:54.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 1 -- 172.21.15.132:0/3996886250 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f079c057cb0 con 0x7f07a80ae380 2024-03-13T07:34:54.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 10 client.4293.objecter ms_dispatch 0x7f07a8000b90 osd_map(25..25 src has 1..25) v4 2024-03-13T07:34:54.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 3 client.4293.objecter handle_osd_map got epochs [25,25] > 0 2024-03-13T07:34:54.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 3 client.4293.objecter handle_osd_map decoding full epoch 25 2024-03-13T07:34:54.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 10 client.4293.objecter checking command 1 2024-03-13T07:34:54.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 1 --2- 172.21.15.132:0/3996886250 >> [v2:172.21.15.132:6818/590522438,v1:172.21.15.132:6819/590522438] conn(0x7f0794073a20 0x7f0794075e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 20 client.4293.objecter _get_session s=0x7f0794073350 osd=0 3 2024-03-13T07:34:54.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 20 client.4293.objecter put_session s=0x7f0794073350 osd=0 3 2024-03-13T07:34:54.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 15 client.4293.objecter _session_command_op_remove -1 1 2024-03-13T07:34:54.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 20 client.4293.objecter _get_session s=0x7f0794073350 osd=0 3 2024-03-13T07:34:54.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 20 client.4293.objecter get_session s=0x7f0794073350 osd=0 3 2024-03-13T07:34:54.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 15 client.4293.objecter _session_command_op_assign 0 1 2024-03-13T07:34:54.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 20 client.4293.objecter put_session s=0x7f0794073350 osd=0 4 2024-03-13T07:34:54.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 10 client.4293.objecter _send_command 1 2024-03-13T07:34:54.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 1 -- 172.21.15.132:0/3996886250 --> [v2:172.21.15.132:6818/590522438,v1:172.21.15.132:6819/590522438] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f079c00cfb0 con 0x7f0794073a20 2024-03-13T07:34:54.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 20 client.4293.objecter dump_active .. 0 homeless 2024-03-13T07:34:54.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 1 -- 172.21.15.132:0/3996886250 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f079c055eb0 con 0x7f07a80ae380 2024-03-13T07:34:54.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.227+0000 7f078ffff640 10 monclient: handle_get_version_reply finishing 1 version 25 2024-03-13T07:34:54.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.228+0000 7f4fcffff640 10 client.4297.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-03-13T07:34:54.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.228+0000 7f4fcffff640 15 client.4297.objecter _session_command_op_assign -1 2 2024-03-13T07:34:54.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.228+0000 7f4fcffff640 20 client.4297.objecter _get_session s=0x7f4fd00732d0 osd=6 3 2024-03-13T07:34:54.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.228+0000 7f4fcffff640 20 client.4297.objecter put_session s=0x7f4fd00732d0 osd=6 3 2024-03-13T07:34:54.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.228+0000 7f4fcffff640 20 client.4297.objecter _get_session s=0x7f4fd00732d0 osd=6 3 2024-03-13T07:34:54.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.228+0000 7f4fcffff640 15 client.4297.objecter _session_command_op_remove -1 2 2024-03-13T07:34:54.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.228+0000 7f4fcffff640 20 client.4297.objecter get_session s=0x7f4fd00732d0 osd=6 3 2024-03-13T07:34:54.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.228+0000 7f4fcffff640 15 client.4297.objecter _session_command_op_assign 6 2 2024-03-13T07:34:54.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.228+0000 7f4fcffff640 20 client.4297.objecter put_session s=0x7f4fd00732d0 osd=6 4 2024-03-13T07:34:54.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.228+0000 7f4fcffff640 10 client.4297.objecter _send_command 2 2024-03-13T07:34:54.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.228+0000 7f4fcffff640 1 -- 172.21.15.132:0/2509614961 --> [v2:172.21.15.195:6800/4106887608,v1:172.21.15.195:6801/4106887608] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4fec1408a0 con 0x7f4fd00739a0 2024-03-13T07:34:54.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.229+0000 7f07a7fff640 1 --2- 172.21.15.132:0/3996886250 >> [v2:172.21.15.132:6818/590522438,v1:172.21.15.132:6819/590522438] conn(0x7f0794073a20 0x7f0794075e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.229+0000 7f07a7fff640 10 monclient: get_auth_request con 0x7f0794073a20 auth_method 0 2024-03-13T07:34:54.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.229+0000 7f692d0f1640 1 --2- 172.21.15.132:0/1569100031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6928072270 0x7f692813baf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6918001e80 tx=0x7f691800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.229+0000 7f691e7fc640 10 client.?.objecter ms_handle_connect 0x7f6928072270 2024-03-13T07:34:54.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.229+0000 7f691e7fc640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:54.260 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.231+0000 7f3ce6cb7640 10 client.4300.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-03-13T07:34:54.260 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.231+0000 7f3ce6cb7640 15 client.4300.objecter _session_command_op_assign -1 2 2024-03-13T07:34:54.260 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.231+0000 7f3ce6cb7640 20 client.4300.objecter _get_session s=0x7f3ccc073320 osd=5 3 2024-03-13T07:34:54.260 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.231+0000 7f3ce6cb7640 20 client.4300.objecter put_session s=0x7f3ccc073320 osd=5 3 2024-03-13T07:34:54.260 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.231+0000 7f3ce6cb7640 20 client.4300.objecter _get_session s=0x7f3ccc073320 osd=5 3 2024-03-13T07:34:54.260 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.231+0000 7f3ce6cb7640 15 client.4300.objecter _session_command_op_remove -1 2 2024-03-13T07:34:54.260 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.231+0000 7f3ce6cb7640 20 client.4300.objecter get_session s=0x7f3ccc073320 osd=5 3 2024-03-13T07:34:54.261 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.231+0000 7f3ce6cb7640 15 client.4300.objecter _session_command_op_assign 5 2 2024-03-13T07:34:54.261 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.231+0000 7f3ce6cb7640 20 client.4300.objecter put_session s=0x7f3ccc073320 osd=5 4 2024-03-13T07:34:54.261 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.231+0000 7f3ce6cb7640 10 client.4300.objecter _send_command 2 2024-03-13T07:34:54.261 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.231+0000 7f3ce6cb7640 1 -- 172.21.15.132:0/733602661 --> [v2:172.21.15.195:6816/1953907706,v1:172.21.15.195:6817/1953907706] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3ce00adfd0 con 0x7f3ccc0739f0 2024-03-13T07:34:54.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f691e7fc640 1 -- 172.21.15.132:0/1569100031 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918018070 con 0x7f6928072270 2024-03-13T07:34:54.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f691e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f691e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:54.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f691e7fc640 10 monclient: dump: 2024-03-13T07:34:54.262 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.262 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.262 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.262 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.262 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.262 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.262 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.263 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.263 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.263 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f691e7fc640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f691e7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f691e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f691e7fc640 1 -- 172.21.15.132:0/1569100031 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69180052f0 con 0x7f6928072270 2024-03-13T07:34:54.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f691e7fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f691e7fc640 1 -- 172.21.15.132:0/1569100031 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918013bc0 con 0x7f6928072270 2024-03-13T07:34:54.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f691e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f691e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:54.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f691e7fc640 10 monclient: dump: 2024-03-13T07:34:54.263 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.263 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.263 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.264 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.264 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.264 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.264 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.264 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.264 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.264 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f692e8f4640 5 monclient: authenticate success, global_id 4296 2024-03-13T07:34:54.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f692e8f4640 10 monclient: _renew_subs 2024-03-13T07:34:54.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f692e8f4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f692e8f4640 1 -- 172.21.15.132:0/1569100031 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69280aa910 con 0x7f6928072270 2024-03-13T07:34:54.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.233+0000 7f691ffff640 4 set_mon_vals no callback set 2024-03-13T07:34:54.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.234+0000 7f692e8f4640 10 client.4296.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.234+0000 7f692e8f4640 10 monclient: _renew_subs 2024-03-13T07:34:54.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.234+0000 7f692e8f4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.234+0000 7f692e8f4640 1 -- 172.21.15.132:0/1569100031 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69280ab010 con 0x7f6928072270 2024-03-13T07:34:54.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.234+0000 7f692e8f4640 1 librados: init done 2024-03-13T07:34:54.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.234+0000 7f692e8f4640 10 client.4296.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.234+0000 7f692e8f4640 15 client.4296.objecter _session_command_op_assign -1 1 2024-03-13T07:34:54.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.234+0000 7f692e8f4640 20 client.4296.objecter _get_session osd=-1 returning homeless 2024-03-13T07:34:54.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.234+0000 7f692e8f4640 10 client.4296.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.234+0000 7f692e8f4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.234+0000 7f692e8f4640 1 -- 172.21.15.132:0/1569100031 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f692806a630 con 0x7f6928072270 2024-03-13T07:34:54.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.235+0000 7f078ffff640 10 client.4293.objecter ms_handle_connect 0x7f0794073a20 2024-03-13T07:34:54.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.235+0000 7f07a7fff640 1 --2- 172.21.15.132:0/3996886250 >> [v2:172.21.15.132:6818/590522438,v1:172.21.15.132:6819/590522438] conn(0x7f0794073a20 0x7f0794075e40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3cdd7fa640 1 -- 172.21.15.132:0/733602661 <== osd.5 v2:172.21.15.195:6816/1953907706 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3ce00adfd0 con 0x7f3ccc0739f0 2024-03-13T07:34:54.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3cdd7fa640 10 client.4300.objecter ms_dispatch 0x7f3ce0000b90 command_reply(tid 2: 0 ) v1 2024-03-13T07:34:54.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3cdd7fa640 10 client.4300.objecter _finish_command 2 = system:0 2024-03-13T07:34:54.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3cdd7fa640 20 client.4300.objecter put_session s=0x7f3ccc073320 osd=5 4 2024-03-13T07:34:54.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3cdd7fa640 15 client.4300.objecter _session_command_op_remove 5 2 2024-03-13T07:34:54.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce6cb7640 10 librados: watch_flush enter 2024-03-13T07:34:54.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce6cb7640 10 librados: watch_flush exit 2024-03-13T07:34:54.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce6cb7640 10 client.4300.objecter close_session for osd.5 2024-03-13T07:34:54.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce6cb7640 1 -- 172.21.15.132:0/733602661 >> [v2:172.21.15.195:6816/1953907706,v1:172.21.15.195:6817/1953907706] conn(0x7f3ccc0739f0 msgr2=0x7f3ccc075e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce6cb7640 1 --2- 172.21.15.132:0/733602661 >> [v2:172.21.15.195:6816/1953907706,v1:172.21.15.195:6817/1953907706] conn(0x7f3ccc0739f0 0x7f3ccc075e10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce6cb7640 20 client.4300.objecter put_session s=0x7f3ccc073320 osd=5 1 2024-03-13T07:34:54.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce6cb7640 20 client.4300.objecter shutdown clearing up homeless session... 2024-03-13T07:34:54.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce6cb7640 10 client.4300.objecter successfully canceled tick 2024-03-13T07:34:54.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce6cb7640 1 -- 172.21.15.132:0/733602661 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3ccc06d5b0 msgr2=0x7f3ccc06fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce6cb7640 1 --2- 172.21.15.132:0/733602661 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3ccc06d5b0 0x7f3ccc06fa70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4006a90 tx=0x7f3cd4006a20 comp rx=0 tx=0).stop 2024-03-13T07:34:54.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce6cb7640 10 monclient: shutdown 2024-03-13T07:34:54.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce6cb7640 20 monclient: shutdown canceling and discarding version request 1 2024-03-13T07:34:54.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce6cb7640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce6cb7640 1 -- 172.21.15.132:0/733602661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ce01538d0 msgr2=0x7f3ce006fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce6cb7640 1 --2- 172.21.15.132:0/733602661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ce01538d0 0x7f3ce006fd80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009d90 tx=0x7f3cd8004a40 comp rx=0 tx=0).stop 2024-03-13T07:34:54.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce5254640 1 -- 172.21.15.132:0/733602661 reap_dead start 2024-03-13T07:34:54.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.237+0000 7f3ce6cb7640 1 -- 172.21.15.132:0/733602661 shutdown_connections 2024-03-13T07:34:54.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.238+0000 7f3ce6cb7640 1 -- 172.21.15.132:0/733602661 >> 172.21.15.132:0/733602661 conn(0x7f3ce00b0660 msgr2=0x7f3ce006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.238+0000 7f3ce6cb7640 1 -- 172.21.15.132:0/733602661 shutdown_connections 2024-03-13T07:34:54.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.238+0000 7f3ce6cb7640 1 -- 172.21.15.132:0/733602661 wait complete. 2024-03-13T07:34:54.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.238+0000 7f3ce6cb7640 1 librados: shutdown 2024-03-13T07:34:54.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.242+0000 7fcc83fff640 1 -- 172.21.15.132:0/3861919449 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc84055e00 con 0x7fcc90031630 2024-03-13T07:34:54.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.242+0000 7fcc83fff640 10 monclient: handle_get_version_reply finishing 1 version 25 2024-03-13T07:34:54.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.244+0000 7f4fe27fc640 1 -- 172.21.15.132:0/2509614961 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fdc0546d0 con 0x7f4fec133fb0 2024-03-13T07:34:54.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.244+0000 7f4fe27fc640 10 monclient: handle_get_version_reply finishing 1 version 25 2024-03-13T07:34:54.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.245+0000 7f4fe27fc640 1 -- 172.21.15.132:0/2509614961 <== osd.6 v2:172.21.15.195:6800/4106887608 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4fec1408a0 con 0x7f4fd00739a0 2024-03-13T07:34:54.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.245+0000 7f4fe27fc640 10 client.4297.objecter ms_dispatch 0x7f4fec000b90 command_reply(tid 2: 0 ) v1 2024-03-13T07:34:54.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.245+0000 7f4fe27fc640 10 client.4297.objecter _finish_command 2 = system:0 2024-03-13T07:34:54.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.245+0000 7f4fe27fc640 20 client.4297.objecter put_session s=0x7f4fd00732d0 osd=6 4 2024-03-13T07:34:54.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.245+0000 7f4fe27fc640 15 client.4297.objecter _session_command_op_remove 6 2 2024-03-13T07:34:54.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.245+0000 7f4ff4205640 10 librados: watch_flush enter 2024-03-13T07:34:54.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.246+0000 7f4ff4205640 10 librados: watch_flush exit 2024-03-13T07:34:54.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.246+0000 7f4ff4205640 10 client.4297.objecter close_session for osd.6 2024-03-13T07:34:54.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.246+0000 7f4ff4205640 1 -- 172.21.15.132:0/2509614961 >> [v2:172.21.15.195:6800/4106887608,v1:172.21.15.195:6801/4106887608] conn(0x7f4fd00739a0 msgr2=0x7f4fd0075dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.246+0000 7f4ff4205640 1 --2- 172.21.15.132:0/2509614961 >> [v2:172.21.15.195:6800/4106887608,v1:172.21.15.195:6801/4106887608] conn(0x7f4fd00739a0 0x7f4fd0075dc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.246+0000 7f4ff4205640 20 client.4297.objecter put_session s=0x7f4fd00732d0 osd=6 1 2024-03-13T07:34:54.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.246+0000 7f4ff4205640 20 client.4297.objecter shutdown clearing up homeless session... 2024-03-13T07:34:54.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.246+0000 7f4ff4205640 10 client.4297.objecter successfully canceled tick 2024-03-13T07:34:54.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.246+0000 7f4ff4205640 1 -- 172.21.15.132:0/2509614961 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4fd006d560 msgr2=0x7f4fd006fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.246+0000 7f4ff4205640 1 --2- 172.21.15.132:0/2509614961 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4fd006d560 0x7f4fd006fa20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400ab90 tx=0x7f4fe4009250 comp rx=0 tx=0).stop 2024-03-13T07:34:54.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.246+0000 7f4ff4205640 10 monclient: shutdown 2024-03-13T07:34:54.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.246+0000 7f4ff4205640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.247+0000 7f4ff4205640 1 -- 172.21.15.132:0/2509614961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fec133fb0 msgr2=0x7f4fec134460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.247+0000 7f4ff4205640 1 --2- 172.21.15.132:0/2509614961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fec133fb0 0x7f4fec134460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00c390 tx=0x7f4fdc00c850 comp rx=0 tx=0).stop 2024-03-13T07:34:54.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.247+0000 7f854f7fe640 1 -- 172.21.15.132:0/4257405221 <== osd.3 v2:172.21.15.132:6810/1770057764 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f85380764c0 con 0x7f85380739f0 2024-03-13T07:34:54.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.247+0000 7f854f7fe640 10 client.4295.objecter ms_dispatch 0x7f8558000b90 command_reply(tid 1: 0 ) v1 2024-03-13T07:34:54.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.247+0000 7f854f7fe640 10 client.4295.objecter _finish_command 1 = system:0 2024-03-13T07:34:54.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.247+0000 7f854f7fe640 20 client.4295.objecter put_session s=0x7f8538073320 osd=3 4 2024-03-13T07:34:54.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.247+0000 7f854f7fe640 15 client.4295.objecter _session_command_op_remove 3 1 2024-03-13T07:34:54.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.247+0000 7f4ff27a2640 1 -- 172.21.15.132:0/2509614961 reap_dead start 2024-03-13T07:34:54.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.248+0000 7f4ff4205640 1 -- 172.21.15.132:0/2509614961 shutdown_connections 2024-03-13T07:34:54.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.248+0000 7f4ff4205640 1 -- 172.21.15.132:0/2509614961 >> 172.21.15.132:0/2509614961 conn(0x7f4fec0b3c40 msgr2=0x7f4fec0af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.248+0000 7f4ff4205640 1 -- 172.21.15.132:0/2509614961 shutdown_connections 2024-03-13T07:34:54.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.248+0000 7f4ff4205640 1 -- 172.21.15.132:0/2509614961 wait complete. 2024-03-13T07:34:54.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.248+0000 7f4ff4205640 1 librados: shutdown 2024-03-13T07:34:54.276 INFO:teuthology.orchestra.run.smithi132.stdout:55834574855 2024-03-13T07:34:54.277 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-13T07:34:54.277 INFO:teuthology.orchestra.run.smithi132.stdout:64424509447 2024-03-13T07:34:54.278 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-13T07:34:54.278 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.264+0000 7fcc83fff640 1 -- 172.21.15.132:0/3861919449 <== osd.4 v2:172.21.15.195:6802/699638467 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcc8400ee50 con 0x7fcc7c0739f0 2024-03-13T07:34:54.278 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.264+0000 7fcc83fff640 10 client.4294.objecter ms_dispatch 0x7fcc98000b90 command_reply(tid 1: 0 ) v1 2024-03-13T07:34:54.278 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.264+0000 7fcc83fff640 10 client.4294.objecter _finish_command 1 = system:0 2024-03-13T07:34:54.278 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.264+0000 7fcc83fff640 20 client.4294.objecter put_session s=0x7fcc7c073320 osd=4 4 2024-03-13T07:34:54.278 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.264+0000 7fcc83fff640 15 client.4294.objecter _session_command_op_remove 4 1 2024-03-13T07:34:54.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.266+0000 7f855d24b640 10 client.4295.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-03-13T07:34:54.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.266+0000 7f855d24b640 15 client.4295.objecter _session_command_op_assign -1 2 2024-03-13T07:34:54.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.266+0000 7f855d24b640 20 client.4295.objecter _get_session s=0x7f8538073320 osd=3 3 2024-03-13T07:34:54.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.266+0000 7f855d24b640 20 client.4295.objecter put_session s=0x7f8538073320 osd=3 3 2024-03-13T07:34:54.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.266+0000 7f855d24b640 20 client.4295.objecter _get_session s=0x7f8538073320 osd=3 3 2024-03-13T07:34:54.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.266+0000 7f855d24b640 15 client.4295.objecter _session_command_op_remove -1 2 2024-03-13T07:34:54.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.266+0000 7f855d24b640 20 client.4295.objecter get_session s=0x7f8538073320 osd=3 3 2024-03-13T07:34:54.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.266+0000 7f855d24b640 15 client.4295.objecter _session_command_op_assign 3 2 2024-03-13T07:34:54.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.266+0000 7f855d24b640 20 client.4295.objecter put_session s=0x7f8538073320 osd=3 4 2024-03-13T07:34:54.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.266+0000 7f855d24b640 10 client.4295.objecter _send_command 2 2024-03-13T07:34:54.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.266+0000 7f855d24b640 1 -- 172.21.15.132:0/4257405221 --> [v2:172.21.15.132:6810/1770057764,v1:172.21.15.132:6811/1770057764] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f85581455c0 con 0x7f85380739f0 2024-03-13T07:34:54.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.266+0000 7f691e7fc640 1 -- 172.21.15.132:0/1569100031 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f69180035f0 con 0x7f6928072270 2024-03-13T07:34:54.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.266+0000 7f691e7fc640 1 --2- 172.21.15.132:0/1569100031 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f690c06d560 0x7f690c06fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.268+0000 7f692d8f2640 1 --2- 172.21.15.132:0/1569100031 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f690c06d560 0x7f690c06fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.268+0000 7f692d8f2640 10 monclient: get_auth_request con 0x7f690c06d560 auth_method 0 2024-03-13T07:34:54.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.268+0000 7f692d8f2640 1 --2- 172.21.15.132:0/1569100031 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f690c06d560 0x7f690c06fa20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f69200012e0 tx=0x7f6920007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.269+0000 7f691e7fc640 10 client.4296.objecter ms_handle_connect 0x7f690c06d560 2024-03-13T07:34:54.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 1 -- 172.21.15.132:0/1569100031 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f691805b240 con 0x7f6928072270 2024-03-13T07:34:54.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 10 client.4296.objecter ms_dispatch 0x7f6928000b90 osd_map(25..25 src has 1..25) v4 2024-03-13T07:34:54.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 3 client.4296.objecter handle_osd_map got epochs [25,25] > 0 2024-03-13T07:34:54.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 3 client.4296.objecter handle_osd_map decoding full epoch 25 2024-03-13T07:34:54.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 10 client.4296.objecter checking command 1 2024-03-13T07:34:54.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 1 --2- 172.21.15.132:0/1569100031 >> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] conn(0x7f690c0739a0 0x7f690c075dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 20 client.4296.objecter _get_session s=0x7f690c0732d0 osd=7 3 2024-03-13T07:34:54.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 20 client.4296.objecter put_session s=0x7f690c0732d0 osd=7 3 2024-03-13T07:34:54.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 15 client.4296.objecter _session_command_op_remove -1 1 2024-03-13T07:34:54.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 20 client.4296.objecter _get_session s=0x7f690c0732d0 osd=7 3 2024-03-13T07:34:54.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 20 client.4296.objecter get_session s=0x7f690c0732d0 osd=7 3 2024-03-13T07:34:54.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 15 client.4296.objecter _session_command_op_assign 7 1 2024-03-13T07:34:54.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 20 client.4296.objecter put_session s=0x7f690c0732d0 osd=7 4 2024-03-13T07:34:54.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 10 client.4296.objecter _send_command 1 2024-03-13T07:34:54.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 1 -- 172.21.15.132:0/1569100031 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f691801ae70 con 0x7f690c0739a0 2024-03-13T07:34:54.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 20 client.4296.objecter dump_active .. 0 homeless 2024-03-13T07:34:54.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 1 -- 172.21.15.132:0/1569100031 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6918059440 con 0x7f6928072270 2024-03-13T07:34:54.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.272+0000 7f691e7fc640 10 monclient: handle_get_version_reply finishing 1 version 25 2024-03-13T07:34:54.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.273+0000 7f692e0f3640 1 --2- 172.21.15.132:0/1569100031 >> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] conn(0x7f690c0739a0 0x7f690c075dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.274+0000 7f692e0f3640 10 monclient: get_auth_request con 0x7f690c0739a0 auth_method 0 2024-03-13T07:34:54.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.277+0000 7f692e0f3640 1 --2- 172.21.15.132:0/1569100031 >> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] conn(0x7f690c0739a0 0x7f690c075dc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.277+0000 7f691e7fc640 10 client.4296.objecter ms_handle_connect 0x7f690c0739a0 2024-03-13T07:34:54.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.284+0000 7fcc9c9a3640 10 client.4294.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-03-13T07:34:54.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.284+0000 7fcc9c9a3640 15 client.4294.objecter _session_command_op_assign -1 2 2024-03-13T07:34:54.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.284+0000 7fcc9c9a3640 20 client.4294.objecter _get_session s=0x7fcc7c073320 osd=4 3 2024-03-13T07:34:54.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.284+0000 7fcc9c9a3640 20 client.4294.objecter put_session s=0x7fcc7c073320 osd=4 3 2024-03-13T07:34:54.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.284+0000 7fcc9c9a3640 20 client.4294.objecter _get_session s=0x7fcc7c073320 osd=4 3 2024-03-13T07:34:54.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.284+0000 7fcc9c9a3640 15 client.4294.objecter _session_command_op_remove -1 2 2024-03-13T07:34:54.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.284+0000 7fcc9c9a3640 20 client.4294.objecter get_session s=0x7fcc7c073320 osd=4 3 2024-03-13T07:34:54.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.284+0000 7fcc9c9a3640 15 client.4294.objecter _session_command_op_assign 4 2 2024-03-13T07:34:54.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.284+0000 7fcc9c9a3640 20 client.4294.objecter put_session s=0x7fcc7c073320 osd=4 4 2024-03-13T07:34:54.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.284+0000 7fcc9c9a3640 10 client.4294.objecter _send_command 2 2024-03-13T07:34:54.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.284+0000 7fcc9c9a3640 1 -- 172.21.15.132:0/3861919449 --> [v2:172.21.15.195:6802/699638467,v1:172.21.15.195:6803/699638467] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcc9002bad0 con 0x7fcc7c0739f0 2024-03-13T07:34:54.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.288+0000 7f854f7fe640 1 -- 172.21.15.132:0/4257405221 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8548078200 con 0x7f85580b0270 2024-03-13T07:34:54.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.288+0000 7f854f7fe640 10 monclient: handle_get_version_reply finishing 1 version 25 2024-03-13T07:34:54.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.291+0000 7f50767fc640 1 -- 172.21.15.132:0/3235253391 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5078078150 con 0x7f50800717c0 2024-03-13T07:34:54.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.291+0000 7f50767fc640 10 monclient: handle_get_version_reply finishing 1 version 25 2024-03-13T07:34:54.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.293+0000 7f854f7fe640 1 -- 172.21.15.132:0/4257405221 <== osd.3 v2:172.21.15.132:6810/1770057764 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f85581455c0 con 0x7f85380739f0 2024-03-13T07:34:54.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.293+0000 7f854f7fe640 10 client.4295.objecter ms_dispatch 0x7f8558000b90 command_reply(tid 2: 0 ) v1 2024-03-13T07:34:54.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.293+0000 7f854f7fe640 10 client.4295.objecter _finish_command 2 = system:0 2024-03-13T07:34:54.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.293+0000 7f854f7fe640 20 client.4295.objecter put_session s=0x7f8538073320 osd=3 4 2024-03-13T07:34:54.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.293+0000 7f854f7fe640 15 client.4295.objecter _session_command_op_remove 3 2 2024-03-13T07:34:54.295 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855d24b640 10 librados: watch_flush enter 2024-03-13T07:34:54.295 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855d24b640 10 librados: watch_flush exit 2024-03-13T07:34:54.295 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855d24b640 10 client.4295.objecter close_session for osd.3 2024-03-13T07:34:54.295 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855d24b640 1 -- 172.21.15.132:0/4257405221 >> [v2:172.21.15.132:6810/1770057764,v1:172.21.15.132:6811/1770057764] conn(0x7f85380739f0 msgr2=0x7f8538075e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.296 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855d24b640 1 --2- 172.21.15.132:0/4257405221 >> [v2:172.21.15.132:6810/1770057764,v1:172.21.15.132:6811/1770057764] conn(0x7f85380739f0 0x7f8538075e10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.296 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855d24b640 20 client.4295.objecter put_session s=0x7f8538073320 osd=3 1 2024-03-13T07:34:54.296 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855d24b640 20 client.4295.objecter shutdown clearing up homeless session... 2024-03-13T07:34:54.296 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855d24b640 10 client.4295.objecter successfully canceled tick 2024-03-13T07:34:54.296 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855d24b640 1 -- 172.21.15.132:0/4257405221 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f853806d5b0 msgr2=0x7f853806fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.296 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855d24b640 1 --2- 172.21.15.132:0/4257405221 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f853806d5b0 0x7f853806fa70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8558143350 tx=0x7f8540009040 comp rx=0 tx=0).stop 2024-03-13T07:34:54.296 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855d24b640 10 monclient: shutdown 2024-03-13T07:34:54.296 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855d24b640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.296 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855d24b640 1 -- 172.21.15.132:0/4257405221 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85580b0270 msgr2=0x7f8558142b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.296 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855d24b640 1 --2- 172.21.15.132:0/4257405221 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85580b0270 0x7f8558142b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8548037040 tx=0x7f8548004a40 comp rx=0 tx=0).stop 2024-03-13T07:34:54.296 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855759e640 1 -- 172.21.15.132:0/4257405221 reap_dead start 2024-03-13T07:34:54.296 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855d24b640 1 -- 172.21.15.132:0/4257405221 shutdown_connections 2024-03-13T07:34:54.296 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7f855d24b640 1 -- 172.21.15.132:0/4257405221 >> 172.21.15.132:0/4257405221 conn(0x7f855806d2b0 msgr2=0x7f85580af7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.297 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7fcc83fff640 1 -- 172.21.15.132:0/3861919449 <== osd.4 v2:172.21.15.195:6802/699638467 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcc9002bad0 con 0x7fcc7c0739f0 2024-03-13T07:34:54.297 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7fcc83fff640 10 client.4294.objecter ms_dispatch 0x7fcc98000b90 command_reply(tid 2: 0 ) v1 2024-03-13T07:34:54.297 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7fcc83fff640 10 client.4294.objecter _finish_command 2 = system:0 2024-03-13T07:34:54.297 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7fcc83fff640 20 client.4294.objecter put_session s=0x7fcc7c073320 osd=4 4 2024-03-13T07:34:54.297 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.294+0000 7fcc83fff640 15 client.4294.objecter _session_command_op_remove 4 2 2024-03-13T07:34:54.297 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7f855d24b640 1 -- 172.21.15.132:0/4257405221 shutdown_connections 2024-03-13T07:34:54.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7f855d24b640 1 -- 172.21.15.132:0/4257405221 wait complete. 2024-03-13T07:34:54.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7f855d24b640 1 librados: shutdown 2024-03-13T07:34:54.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7fcc9c9a3640 10 librados: watch_flush enter 2024-03-13T07:34:54.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7fcc9c9a3640 10 librados: watch_flush exit 2024-03-13T07:34:54.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7fcc9c9a3640 10 client.4294.objecter close_session for osd.4 2024-03-13T07:34:54.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7fcc9c9a3640 1 -- 172.21.15.132:0/3861919449 >> [v2:172.21.15.195:6802/699638467,v1:172.21.15.195:6803/699638467] conn(0x7fcc7c0739f0 msgr2=0x7fcc7c075e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7fcc9c9a3640 1 --2- 172.21.15.132:0/3861919449 >> [v2:172.21.15.195:6802/699638467,v1:172.21.15.195:6803/699638467] conn(0x7fcc7c0739f0 0x7fcc7c075e10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7fcc9c9a3640 20 client.4294.objecter put_session s=0x7fcc7c073320 osd=4 1 2024-03-13T07:34:54.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7fcc9c9a3640 20 client.4294.objecter shutdown clearing up homeless session... 2024-03-13T07:34:54.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7fcc9c9a3640 10 client.4294.objecter successfully canceled tick 2024-03-13T07:34:54.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7fcc9c9a3640 1 -- 172.21.15.132:0/3861919449 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcc7c06d5b0 msgr2=0x7fcc7c06fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7fcc9c9a3640 1 --2- 172.21.15.132:0/3861919449 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcc7c06d5b0 0x7fcc7c06fa70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c036000 tx=0x7fcc8c030000 comp rx=0 tx=0).stop 2024-03-13T07:34:54.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7fcc9c9a3640 10 monclient: shutdown 2024-03-13T07:34:54.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7fcc9c9a3640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7fcc9c9a3640 1 -- 172.21.15.132:0/3861919449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc90031630 msgr2=0x7fcc900c1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7fcc9c9a3640 1 --2- 172.21.15.132:0/3861919449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc90031630 0x7fcc900c1a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400d5d0 tx=0x7fcc8400da90 comp rx=0 tx=0).stop 2024-03-13T07:34:54.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.295+0000 7fcc9dc05640 1 -- 172.21.15.132:0/3861919449 reap_dead start 2024-03-13T07:34:54.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.297+0000 7fcc9c9a3640 1 -- 172.21.15.132:0/3861919449 shutdown_connections 2024-03-13T07:34:54.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.297+0000 7fcc9c9a3640 1 -- 172.21.15.132:0/3861919449 >> 172.21.15.132:0/3861919449 conn(0x7fcc90006070 msgr2=0x7fcc9002e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.298+0000 7fcc9c9a3640 1 -- 172.21.15.132:0/3861919449 shutdown_connections 2024-03-13T07:34:54.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.298+0000 7fcc9c9a3640 1 -- 172.21.15.132:0/3861919449 wait complete. 2024-03-13T07:34:54.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.298+0000 7fcc9c9a3640 1 librados: shutdown 2024-03-13T07:34:54.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.305+0000 7f50767fc640 1 -- 172.21.15.132:0/3235253391 <== osd.2 v2:172.21.15.132:6826/958254560 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5078045e70 con 0x7f5064073a20 2024-03-13T07:34:54.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.305+0000 7f50767fc640 10 client.4292.objecter ms_dispatch 0x7f5080000b90 command_reply(tid 1: 0 ) v1 2024-03-13T07:34:54.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.305+0000 7f50767fc640 10 client.4292.objecter _finish_command 1 = system:0 2024-03-13T07:34:54.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.305+0000 7f50767fc640 20 client.4292.objecter put_session s=0x7f5064073350 osd=2 4 2024-03-13T07:34:54.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.305+0000 7f50767fc640 15 client.4292.objecter _session_command_op_remove 2 1 2024-03-13T07:34:54.317 INFO:teuthology.orchestra.run.smithi132.stdout:77309411338 2024-03-13T07:34:54.317 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-13T07:34:54.318 INFO:teuthology.orchestra.run.smithi132.stdout:68719476742 2024-03-13T07:34:54.318 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-13T07:34:54.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.328+0000 7f691e7fc640 1 -- 172.21.15.132:0/1569100031 <== osd.7 v2:172.21.15.195:6820/3378300314 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f691801ae70 con 0x7f690c0739a0 2024-03-13T07:34:54.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.328+0000 7f691e7fc640 10 client.4296.objecter ms_dispatch 0x7f6928000b90 command_reply(tid 1: 0 ) v1 2024-03-13T07:34:54.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.328+0000 7f691e7fc640 10 client.4296.objecter _finish_command 1 = system:0 2024-03-13T07:34:54.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.328+0000 7f691e7fc640 20 client.4296.objecter put_session s=0x7f690c0732d0 osd=7 4 2024-03-13T07:34:54.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.328+0000 7f691e7fc640 15 client.4296.objecter _session_command_op_remove 7 1 2024-03-13T07:34:54.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.336+0000 7fb88d7fa640 1 -- 172.21.15.132:0/2155661044 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb87c007210 con 0x7fb88802ec20 2024-03-13T07:34:54.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.336+0000 7fb88d7fa640 1 --2- 172.21.15.132:0/2155661044 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb87406d5b0 0x7fb87406fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.338+0000 7f078ffff640 1 -- 172.21.15.132:0/3996886250 <== osd.0 v2:172.21.15.132:6818/590522438 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f079c00cfb0 con 0x7f0794073a20 2024-03-13T07:34:54.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.338+0000 7f078ffff640 10 client.4293.objecter ms_dispatch 0x7f07a8000b90 command_reply(tid 1: 0 ) v1 2024-03-13T07:34:54.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.338+0000 7f078ffff640 10 client.4293.objecter _finish_command 1 = system:0 2024-03-13T07:34:54.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.338+0000 7f078ffff640 20 client.4293.objecter put_session s=0x7f0794073350 osd=0 4 2024-03-13T07:34:54.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.338+0000 7f078ffff640 15 client.4293.objecter _session_command_op_remove 0 1 2024-03-13T07:34:54.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.339+0000 7fb8949b9640 1 --2- 172.21.15.132:0/2155661044 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb87406d5b0 0x7fb87406fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.339+0000 7fb8949b9640 10 monclient: get_auth_request con 0x7fb87406d5b0 auth_method 0 2024-03-13T07:34:54.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.340+0000 7fb8949b9640 1 --2- 172.21.15.132:0/2155661044 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb87406d5b0 0x7fb87406fa70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb884006c90 tx=0x7fb884006c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.340+0000 7fb88d7fa640 10 client.4298.objecter ms_handle_connect 0x7fb87406d5b0 2024-03-13T07:34:54.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.337+0000 7f5087bac640 10 client.4292.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-03-13T07:34:54.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.337+0000 7f5087bac640 15 client.4292.objecter _session_command_op_assign -1 2 2024-03-13T07:34:54.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.337+0000 7f5087bac640 20 client.4292.objecter _get_session s=0x7f5064073350 osd=2 3 2024-03-13T07:34:54.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.337+0000 7f5087bac640 20 client.4292.objecter put_session s=0x7f5064073350 osd=2 3 2024-03-13T07:34:54.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.337+0000 7f5087bac640 20 client.4292.objecter _get_session s=0x7f5064073350 osd=2 3 2024-03-13T07:34:54.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.337+0000 7f5087bac640 15 client.4292.objecter _session_command_op_remove -1 2 2024-03-13T07:34:54.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.337+0000 7f5087bac640 20 client.4292.objecter get_session s=0x7f5064073350 osd=2 3 2024-03-13T07:34:54.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.337+0000 7f5087bac640 15 client.4292.objecter _session_command_op_assign 2 2 2024-03-13T07:34:54.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.337+0000 7f5087bac640 20 client.4292.objecter put_session s=0x7f5064073350 osd=2 4 2024-03-13T07:34:54.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.337+0000 7f5087bac640 10 client.4292.objecter _send_command 2 2024-03-13T07:34:54.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.337+0000 7f5087bac640 1 -- 172.21.15.132:0/3235253391 --> [v2:172.21.15.132:6826/958254560,v1:172.21.15.132:6827/958254560] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f50800adaa0 con 0x7f5064073a20 2024-03-13T07:34:54.349 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.349+0000 7fb88d7fa640 1 -- 172.21.15.132:0/2155661044 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb87c078b00 con 0x7fb88802ec20 2024-03-13T07:34:54.349 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.349+0000 7fb88d7fa640 10 client.4298.objecter ms_dispatch 0x7fb890000b90 osd_map(25..25 src has 1..25) v4 2024-03-13T07:34:54.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.349+0000 7fb88d7fa640 3 client.4298.objecter handle_osd_map got epochs [25,25] > 0 2024-03-13T07:34:54.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.349+0000 7fb88d7fa640 3 client.4298.objecter handle_osd_map decoding full epoch 25 2024-03-13T07:34:54.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.349+0000 7fb88d7fa640 10 client.4298.objecter checking command 1 2024-03-13T07:34:54.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.349+0000 7fb88d7fa640 1 --2- 172.21.15.132:0/2155661044 >> [v2:172.21.15.132:6802/1071068206,v1:172.21.15.132:6803/1071068206] conn(0x7fb8740739f0 0x7fb874075e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.349+0000 7fb88d7fa640 20 client.4298.objecter _get_session s=0x7fb874073320 osd=1 3 2024-03-13T07:34:54.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.349+0000 7fb88d7fa640 20 client.4298.objecter put_session s=0x7fb874073320 osd=1 3 2024-03-13T07:34:54.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.349+0000 7fb88d7fa640 15 client.4298.objecter _session_command_op_remove -1 1 2024-03-13T07:34:54.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.349+0000 7fb88d7fa640 20 client.4298.objecter _get_session s=0x7fb874073320 osd=1 3 2024-03-13T07:34:54.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.349+0000 7fb88d7fa640 20 client.4298.objecter get_session s=0x7fb874073320 osd=1 3 2024-03-13T07:34:54.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.349+0000 7fb88d7fa640 15 client.4298.objecter _session_command_op_assign 1 1 2024-03-13T07:34:54.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.349+0000 7fb88d7fa640 20 client.4298.objecter put_session s=0x7fb874073320 osd=1 4 2024-03-13T07:34:54.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.349+0000 7fb88d7fa640 10 client.4298.objecter _send_command 1 2024-03-13T07:34:54.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.349+0000 7fb88d7fa640 1 -- 172.21.15.132:0/2155661044 --> [v2:172.21.15.132:6802/1071068206,v1:172.21.15.132:6803/1071068206] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb87c006e50 con 0x7fb8740739f0 2024-03-13T07:34:54.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.349+0000 7fb88d7fa640 20 client.4298.objecter dump_active .. 0 homeless 2024-03-13T07:34:54.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.350+0000 7fb8951ba640 1 --2- 172.21.15.132:0/2155661044 >> [v2:172.21.15.132:6802/1071068206,v1:172.21.15.132:6803/1071068206] conn(0x7fb8740739f0 0x7fb874075e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.350+0000 7fb8951ba640 10 monclient: get_auth_request con 0x7fb8740739f0 auth_method 0 2024-03-13T07:34:54.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.356+0000 7fb8951ba640 1 --2- 172.21.15.132:0/2155661044 >> [v2:172.21.15.132:6802/1071068206,v1:172.21.15.132:6803/1071068206] conn(0x7fb8740739f0 0x7fb874075e10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.356+0000 7fb88d7fa640 10 client.4298.objecter ms_handle_connect 0x7fb8740739f0 2024-03-13T07:34:54.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.365+0000 7f50767fc640 1 -- 172.21.15.132:0/3235253391 <== osd.2 v2:172.21.15.132:6826/958254560 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f50800adaa0 con 0x7f5064073a20 2024-03-13T07:34:54.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.365+0000 7f50767fc640 10 client.4292.objecter ms_dispatch 0x7f5080000b90 command_reply(tid 2: 0 ) v1 2024-03-13T07:34:54.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.365+0000 7f50767fc640 10 client.4292.objecter _finish_command 2 = system:0 2024-03-13T07:34:54.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.365+0000 7f50767fc640 20 client.4292.objecter put_session s=0x7f5064073350 osd=2 4 2024-03-13T07:34:54.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.365+0000 7f50767fc640 15 client.4292.objecter _session_command_op_remove 2 2 2024-03-13T07:34:54.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f07acba3640 10 client.4293.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-03-13T07:34:54.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f07acba3640 15 client.4293.objecter _session_command_op_assign -1 2 2024-03-13T07:34:54.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f07acba3640 20 client.4293.objecter _get_session s=0x7f0794073350 osd=0 3 2024-03-13T07:34:54.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f07acba3640 20 client.4293.objecter put_session s=0x7f0794073350 osd=0 3 2024-03-13T07:34:54.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f07acba3640 20 client.4293.objecter _get_session s=0x7f0794073350 osd=0 3 2024-03-13T07:34:54.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f07acba3640 15 client.4293.objecter _session_command_op_remove -1 2 2024-03-13T07:34:54.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f07acba3640 20 client.4293.objecter get_session s=0x7f0794073350 osd=0 3 2024-03-13T07:34:54.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f07acba3640 15 client.4293.objecter _session_command_op_assign 0 2 2024-03-13T07:34:54.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f07acba3640 20 client.4293.objecter put_session s=0x7f0794073350 osd=0 4 2024-03-13T07:34:54.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f07acba3640 10 client.4293.objecter _send_command 2 2024-03-13T07:34:54.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f07acba3640 1 -- 172.21.15.132:0/3996886250 --> [v2:172.21.15.132:6818/590522438,v1:172.21.15.132:6819/590522438] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f07a8128950 con 0x7f0794073a20 2024-03-13T07:34:54.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.368+0000 7f5087bac640 10 librados: watch_flush enter 2024-03-13T07:34:54.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.368+0000 7f5087bac640 10 librados: watch_flush exit 2024-03-13T07:34:54.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.368+0000 7f5087bac640 10 client.4292.objecter close_session for osd.2 2024-03-13T07:34:54.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.368+0000 7f5087bac640 1 -- 172.21.15.132:0/3235253391 >> [v2:172.21.15.132:6826/958254560,v1:172.21.15.132:6827/958254560] conn(0x7f5064073a20 msgr2=0x7f5064075e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.368+0000 7f5087bac640 1 --2- 172.21.15.132:0/3235253391 >> [v2:172.21.15.132:6826/958254560,v1:172.21.15.132:6827/958254560] conn(0x7f5064073a20 0x7f5064075e40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.368+0000 7f5087bac640 20 client.4292.objecter put_session s=0x7f5064073350 osd=2 1 2024-03-13T07:34:54.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.368+0000 7f5087bac640 20 client.4292.objecter shutdown clearing up homeless session... 2024-03-13T07:34:54.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.368+0000 7f5087bac640 10 client.4292.objecter successfully canceled tick 2024-03-13T07:34:54.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f692e8f4640 10 client.4296.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-03-13T07:34:54.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f692e8f4640 15 client.4296.objecter _session_command_op_assign -1 2 2024-03-13T07:34:54.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f692e8f4640 20 client.4296.objecter _get_session s=0x7f690c0732d0 osd=7 3 2024-03-13T07:34:54.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f692e8f4640 20 client.4296.objecter put_session s=0x7f690c0732d0 osd=7 3 2024-03-13T07:34:54.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f692e8f4640 20 client.4296.objecter _get_session s=0x7f690c0732d0 osd=7 3 2024-03-13T07:34:54.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f692e8f4640 15 client.4296.objecter _session_command_op_remove -1 2 2024-03-13T07:34:54.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f692e8f4640 20 client.4296.objecter get_session s=0x7f690c0732d0 osd=7 3 2024-03-13T07:34:54.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f692e8f4640 15 client.4296.objecter _session_command_op_assign 7 2 2024-03-13T07:34:54.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f692e8f4640 20 client.4296.objecter put_session s=0x7f690c0732d0 osd=7 4 2024-03-13T07:34:54.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f692e8f4640 10 client.4296.objecter _send_command 2 2024-03-13T07:34:54.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.369+0000 7f692e8f4640 1 -- 172.21.15.132:0/1569100031 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f69280b42d0 con 0x7f690c0739a0 2024-03-13T07:34:54.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.370+0000 7f5087bac640 1 -- 172.21.15.132:0/3235253391 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f506406d5e0 msgr2=0x7f506406faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.370+0000 7f5087bac640 1 --2- 172.21.15.132:0/3235253391 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f506406d5e0 0x7f506406faa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5070001560 tx=0x7f5070007990 comp rx=0 tx=0).stop 2024-03-13T07:34:54.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.370+0000 7f5087bac640 10 monclient: shutdown 2024-03-13T07:34:54.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.371+0000 7f5087bac640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.371+0000 7f5087bac640 1 -- 172.21.15.132:0/3235253391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50800717c0 msgr2=0x7f50801372f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.371+0000 7f5087bac640 1 --2- 172.21.15.132:0/3235253391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50800717c0 0x7f50801372f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5078038040 tx=0x7f5078004a40 comp rx=0 tx=0).stop 2024-03-13T07:34:54.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.371+0000 7f5086149640 1 -- 172.21.15.132:0/3235253391 reap_dead start 2024-03-13T07:34:54.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.371+0000 7f5087bac640 1 -- 172.21.15.132:0/3235253391 shutdown_connections 2024-03-13T07:34:54.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.371+0000 7f5087bac640 1 -- 172.21.15.132:0/3235253391 >> 172.21.15.132:0/3235253391 conn(0x7f50800b3c40 msgr2=0x7f50800b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.371+0000 7f5087bac640 1 -- 172.21.15.132:0/3235253391 shutdown_connections 2024-03-13T07:34:54.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.371+0000 7f5087bac640 1 -- 172.21.15.132:0/3235253391 wait complete. 2024-03-13T07:34:54.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.371+0000 7f5087bac640 1 librados: shutdown 2024-03-13T07:34:54.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.374+0000 7f691e7fc640 1 -- 172.21.15.132:0/1569100031 <== osd.7 v2:172.21.15.195:6820/3378300314 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f69280b42d0 con 0x7f690c0739a0 2024-03-13T07:34:54.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.374+0000 7f691e7fc640 10 client.4296.objecter ms_dispatch 0x7f6928000b90 command_reply(tid 2: 0 ) v1 2024-03-13T07:34:54.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.374+0000 7f691e7fc640 10 client.4296.objecter _finish_command 2 = system:0 2024-03-13T07:34:54.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.374+0000 7f691e7fc640 20 client.4296.objecter put_session s=0x7f690c0732d0 osd=7 4 2024-03-13T07:34:54.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.374+0000 7f691e7fc640 15 client.4296.objecter _session_command_op_remove 7 2 2024-03-13T07:34:54.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.382+0000 7f692e8f4640 10 librados: watch_flush enter 2024-03-13T07:34:54.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.382+0000 7f692e8f4640 10 librados: watch_flush exit 2024-03-13T07:34:54.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.382+0000 7f692e8f4640 10 client.4296.objecter close_session for osd.7 2024-03-13T07:34:54.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.382+0000 7f692e8f4640 1 -- 172.21.15.132:0/1569100031 >> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] conn(0x7f690c0739a0 msgr2=0x7f690c075dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.382+0000 7f692e8f4640 1 --2- 172.21.15.132:0/1569100031 >> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] conn(0x7f690c0739a0 0x7f690c075dc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.382+0000 7f692e8f4640 20 client.4296.objecter put_session s=0x7f690c0732d0 osd=7 1 2024-03-13T07:34:54.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.382+0000 7f692e8f4640 20 client.4296.objecter shutdown clearing up homeless session... 2024-03-13T07:34:54.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.382+0000 7f692e8f4640 10 client.4296.objecter successfully canceled tick 2024-03-13T07:34:54.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.382+0000 7f692e8f4640 1 -- 172.21.15.132:0/1569100031 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f690c06d560 msgr2=0x7f690c06fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.382+0000 7f692e8f4640 1 --2- 172.21.15.132:0/1569100031 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f690c06d560 0x7f690c06fa20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f69200012e0 tx=0x7f6920007d30 comp rx=0 tx=0).stop 2024-03-13T07:34:54.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.383+0000 7f692e8f4640 10 monclient: shutdown 2024-03-13T07:34:54.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.383+0000 7f692e8f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.383+0000 7f692e8f4640 1 -- 172.21.15.132:0/1569100031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6928072270 msgr2=0x7f692813baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.383+0000 7f692e8f4640 1 --2- 172.21.15.132:0/1569100031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6928072270 0x7f692813baf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6918001e80 tx=0x7f691800c850 comp rx=0 tx=0).stop 2024-03-13T07:34:54.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.383+0000 7f692e0f3640 1 -- 172.21.15.132:0/1569100031 reap_dead start 2024-03-13T07:34:54.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.387+0000 7f692e8f4640 1 -- 172.21.15.132:0/1569100031 shutdown_connections 2024-03-13T07:34:54.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.387+0000 7f692e8f4640 1 -- 172.21.15.132:0/1569100031 >> 172.21.15.132:0/1569100031 conn(0x7f69280b4d60 msgr2=0x7f69280afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.387+0000 7f692e8f4640 1 -- 172.21.15.132:0/1569100031 shutdown_connections 2024-03-13T07:34:54.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.387+0000 7f692e8f4640 1 -- 172.21.15.132:0/1569100031 wait complete. 2024-03-13T07:34:54.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.387+0000 7f692e8f4640 1 librados: shutdown 2024-03-13T07:34:54.399 INFO:teuthology.orchestra.run.smithi132.stdout:85899345927 2024-03-13T07:34:54.400 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-13T07:34:54.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.404+0000 7f078ffff640 1 -- 172.21.15.132:0/3996886250 <== osd.0 v2:172.21.15.132:6818/590522438 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f07a8128950 con 0x7f0794073a20 2024-03-13T07:34:54.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.404+0000 7f078ffff640 10 client.4293.objecter ms_dispatch 0x7f07a8000b90 command_reply(tid 2: 0 ) v1 2024-03-13T07:34:54.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.404+0000 7f078ffff640 10 client.4293.objecter _finish_command 2 = system:0 2024-03-13T07:34:54.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.404+0000 7f078ffff640 20 client.4293.objecter put_session s=0x7f0794073350 osd=0 4 2024-03-13T07:34:54.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.404+0000 7f078ffff640 15 client.4293.objecter _session_command_op_remove 0 2 2024-03-13T07:34:54.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.405+0000 7f07acba3640 10 librados: watch_flush enter 2024-03-13T07:34:54.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.405+0000 7f07acba3640 10 librados: watch_flush exit 2024-03-13T07:34:54.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.405+0000 7f07acba3640 10 client.4293.objecter close_session for osd.0 2024-03-13T07:34:54.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.405+0000 7f07acba3640 1 -- 172.21.15.132:0/3996886250 >> [v2:172.21.15.132:6818/590522438,v1:172.21.15.132:6819/590522438] conn(0x7f0794073a20 msgr2=0x7f0794075e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.405+0000 7f07acba3640 1 --2- 172.21.15.132:0/3996886250 >> [v2:172.21.15.132:6818/590522438,v1:172.21.15.132:6819/590522438] conn(0x7f0794073a20 0x7f0794075e40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.405+0000 7f07acba3640 20 client.4293.objecter put_session s=0x7f0794073350 osd=0 1 2024-03-13T07:34:54.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.405+0000 7f07acba3640 20 client.4293.objecter shutdown clearing up homeless session... 2024-03-13T07:34:54.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.405+0000 7f07acba3640 10 client.4293.objecter successfully canceled tick 2024-03-13T07:34:54.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.409+0000 7f07acba3640 1 -- 172.21.15.132:0/3996886250 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f079406d5e0 msgr2=0x7f079406faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.409+0000 7f07acba3640 1 --2- 172.21.15.132:0/3996886250 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f079406d5e0 0x7f079406faa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0798001560 tx=0x7f0798007990 comp rx=0 tx=0).stop 2024-03-13T07:34:54.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.409+0000 7f07acba3640 10 monclient: shutdown 2024-03-13T07:34:54.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.409+0000 7f07acba3640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.409+0000 7f07acba3640 1 -- 172.21.15.132:0/3996886250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07a80ae380 msgr2=0x7f07a8145410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.409+0000 7f07acba3640 1 --2- 172.21.15.132:0/3996886250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07a80ae380 0x7f07a8145410 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f079c00d800 tx=0x7f079c00dcc0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.411+0000 7f07a7fff640 1 -- 172.21.15.132:0/3996886250 reap_dead start 2024-03-13T07:34:54.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.411+0000 7f07acba3640 1 -- 172.21.15.132:0/3996886250 shutdown_connections 2024-03-13T07:34:54.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.411+0000 7f07acba3640 1 -- 172.21.15.132:0/3996886250 >> 172.21.15.132:0/3996886250 conn(0x7f07a80b08a0 msgr2=0x7f07a80b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.411+0000 7f07acba3640 1 -- 172.21.15.132:0/3996886250 shutdown_connections 2024-03-13T07:34:54.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.411+0000 7f07acba3640 1 -- 172.21.15.132:0/3996886250 wait complete. 2024-03-13T07:34:54.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.412+0000 7f07acba3640 1 librados: shutdown 2024-03-13T07:34:54.417 INFO:teuthology.orchestra.run.smithi132.stdout:55834574854 2024-03-13T07:34:54.417 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-13T07:34:54.436 INFO:teuthology.orchestra.run.smithi132.stdout:85899345928 2024-03-13T07:34:54.436 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-13T07:34:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.460+0000 7f3e44ec1640 10 monclient: get_monmap_and_config 2024-03-13T07:34:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.460+0000 7f3e44ec1640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.460+0000 7f3e44ec1640 10 monclient: monmap: 2024-03-13T07:34:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.461 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:54.462012+0000 2024-03-13T07:34:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:54.462012+0000 2024-03-13T07:34:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:54.462 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:54.462 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.462+0000 7fcf14e4b640 10 monclient: get_monmap_and_config 2024-03-13T07:34:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.462+0000 7fcf14e4b640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.462+0000 7fcf14e4b640 10 monclient: monmap: 2024-03-13T07:34:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:54.464271+0000 2024-03-13T07:34:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:54.464271+0000 2024-03-13T07:34:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:54.464 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:54.464 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.463+0000 7f3e44ec1640 1 Processor -- start 2024-03-13T07:34:54.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.463+0000 7f3e44ec1640 1 -- start start 2024-03-13T07:34:54.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.463+0000 7f3e44ec1640 10 monclient: init 2024-03-13T07:34:54.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.464+0000 7f3e44ec1640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.464+0000 7f3e44ec1640 10 monclient: _add_conns ranks=[0,1,3,4,2] 2024-03-13T07:34:54.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.464+0000 7f3e44ec1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3e4006e610 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:54.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.464+0000 7f3e44ec1640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3e4006f950 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:54.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.464+0000 7f3e44ec1640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3e40144340 0x7f3e40152290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.464+0000 7f3e44ec1640 10 monclient(hunting): picked mon.noname-a con 0x7f3e40144340 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:54.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.464+0000 7f3e44ec1640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e40146a10 con 0x7f3e4006e610 2024-03-13T07:34:54.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.464+0000 7f3e44ec1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e400708d0 con 0x7f3e4006f950 2024-03-13T07:34:54.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.464+0000 7f3e44ec1640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.464+0000 7f3e44ec1640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e40070bb0 con 0x7f3e40144340 2024-03-13T07:34:54.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.464+0000 7f3e44ec1640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.464+0000 7f3e44ec1640 10 monclient(hunting): authenticate will time out at 1321.873771s 2024-03-13T07:34:54.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.464+0000 7fcf14e4b640 1 Processor -- start 2024-03-13T07:34:54.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.464+0000 7fcf14e4b640 1 -- start start 2024-03-13T07:34:54.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.464+0000 7fcf14e4b640 10 monclient: init 2024-03-13T07:34:54.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.465+0000 7fcf14e4b640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.465+0000 7fcf14e4b640 10 monclient: _add_conns ranks=[3,0,4,1,2] 2024-03-13T07:34:54.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.465+0000 7fcf14e4b640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcf1006ead0 0x7fcf1013b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.465+0000 7fcf14e4b640 10 monclient(hunting): picked mon.noname-a con 0x7fcf1006ead0 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:54.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.465+0000 7fcf14e4b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcf1013bbb0 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:54.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.465+0000 7fcf14e4b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcf10071d30 0x7fcf10148f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.465+0000 7fcf14e4b640 10 monclient(hunting): picked mon.noname-c con 0x7fcf10071d30 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:54.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.465+0000 7fcf14e4b640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf10070760 con 0x7fcf1013bbb0 2024-03-13T07:34:54.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.465+0000 7fcf14e4b640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.465+0000 7fcf14e4b640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf1006eeb0 con 0x7fcf1006ead0 2024-03-13T07:34:54.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.465+0000 7fcf14e4b640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.465+0000 7fcf14e4b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf1006fa80 con 0x7fcf10071d30 2024-03-13T07:34:54.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.465+0000 7fcf14e4b640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.465+0000 7fcf14e4b640 10 monclient(hunting): authenticate will time out at 1321.874369s 2024-03-13T07:34:54.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.466+0000 7fcf0ed9d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcf10071d30 0x7fcf10148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.466+0000 7fcf0ed9d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcf10071d30 0x7fcf10148f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:48904/0 (socket says 172.21.15.132:48904) 2024-03-13T07:34:54.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.466+0000 7fcf0ed9d640 1 -- 172.21.15.132:0/1596915602 learned_addr learned my addr 172.21.15.132:0/1596915602 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.466+0000 7fcf0ed9d640 10 monclient(hunting): get_auth_request con 0x7fcf10071d30 auth_method 0 2024-03-13T07:34:54.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.466+0000 7fcf0ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.466+0000 7fcf0ed9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.466+0000 7fcf0ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.467+0000 7fcf0ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.467+0000 7fcf0ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.467+0000 7fcf0ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.468+0000 7f3e3ed9d640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3e40144340 0x7f3e40152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.469+0000 7fcf0ed9d640 10 monclient(hunting): handle_auth_done global_id 4306 payload 274 2024-03-13T07:34:54.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.469+0000 7fcf0ed9d640 1 -- 172.21.15.132:0/1596915602 >> v2:172.21.15.132:3300/0 conn(0x7fcf1006ead0 msgr2=0x7fcf1013b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.469+0000 7fcf0ed9d640 1 --2- 172.21.15.132:0/1596915602 >> v2:172.21.15.132:3300/0 conn(0x7fcf1006ead0 0x7fcf1013b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.469+0000 7fcf0ed9d640 1 -- 172.21.15.132:0/1596915602 >> v1:172.21.15.132:6789/0 conn(0x7fcf1013bbb0 legacy=0x7fcf10071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.469+0000 7fcf0ed9d640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.469+0000 7fcf0ed9d640 1 monclient: found mon.noname-c 2024-03-13T07:34:54.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.469+0000 7fcf0ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.469+0000 7fcf0ed9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.469+0000 7fcf0ed9d640 1 -- 172.21.15.132:0/1596915602 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf10070a40 con 0x7fcf10071d30 2024-03-13T07:34:54.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.470+0000 7f3e3e59c640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f3e4006e610 0x7f3e40143c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:36058/0 (socket says 172.21.15.132:36058) 2024-03-13T07:34:54.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.470+0000 7f3e3e59c640 1 -- 172.21.15.132:0/90752748 learned_addr learned my addr 172.21.15.132:0/90752748 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:34:54.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.471+0000 7fcf0ed9d640 1 --2- 172.21.15.132:0/1596915602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf10071d30 0x7fcf10148f30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcf100730f0 tx=0x7fcf0002cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cf85f9bd8bad33d server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.472+0000 7fb88d7fa640 1 -- 172.21.15.132:0/2155661044 <== osd.1 v2:172.21.15.132:6802/1071068206 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb87c006e50 con 0x7fb8740739f0 2024-03-13T07:34:54.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.472+0000 7fb88d7fa640 10 client.4298.objecter ms_dispatch 0x7fb890000b90 command_reply(tid 1: 0 ) v1 2024-03-13T07:34:54.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.472+0000 7fb88d7fa640 10 client.4298.objecter _finish_command 1 = system:0 2024-03-13T07:34:54.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.472+0000 7fb88d7fa640 20 client.4298.objecter put_session s=0x7fb874073320 osd=1 4 2024-03-13T07:34:54.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.472+0000 7fb88d7fa640 15 client.4298.objecter _session_command_op_remove 1 1 2024-03-13T07:34:54.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.475+0000 7f3e3ed9d640 10 monclient(hunting): get_auth_request con 0x7f3e40144340 auth_method 0 2024-03-13T07:34:54.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.475+0000 7f3e3ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.475+0000 7f3e3ed9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.475+0000 7f3e3ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.476+0000 7fcf0d59a640 1 -- 172.21.15.132:0/1596915602 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0003a070 con 0x7fcf10071d30 2024-03-13T07:34:54.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.477+0000 7fcf0d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.477+0000 7fcf0d59a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:54.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.477+0000 7fcf0d59a640 10 monclient: dump: 2024-03-13T07:34:54.478 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.478 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.478 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.478 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.478 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.479 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.479 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.479 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.479 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.479 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.477+0000 7fcf0d59a640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.477+0000 7fcf0d59a640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.477+0000 7fcf0d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.477+0000 7fcf14e4b640 5 monclient: authenticate success, global_id 4306 2024-03-13T07:34:54.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.477+0000 7fcf14e4b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:54.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.477+0000 7fcf0d59a640 1 -- 172.21.15.132:0/1596915602 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0002db20 con 0x7fcf10071d30 2024-03-13T07:34:54.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.477+0000 7fcf0d59a640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.477+0000 7fcf14e4b640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:54.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.477+0000 7fcf14e4b640 4 set_mon_vals no callback set 2024-03-13T07:34:54.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.478+0000 7fcf14e4b640 10 monclient: shutdown 2024-03-13T07:34:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.478+0000 7fcf14e4b640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.478+0000 7fcf14e4b640 1 -- 172.21.15.132:0/1596915602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf10071d30 msgr2=0x7fcf10148f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.478+0000 7fcf14e4b640 1 --2- 172.21.15.132:0/1596915602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf10071d30 0x7fcf10148f30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcf100730f0 tx=0x7fcf0002cd70 comp rx=0 tx=0).stop 2024-03-13T07:34:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.478+0000 7fcf14e4b640 1 -- 172.21.15.132:0/1596915602 shutdown_connections 2024-03-13T07:34:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.478+0000 7fcf14e4b640 1 --2- 172.21.15.132:0/1596915602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf10071d30 0x7fcf10148f30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.478+0000 7fcf14e4b640 1 --2- 172.21.15.132:0/1596915602 >> v2:172.21.15.132:3300/0 conn(0x7fcf1006ead0 0x7fcf1013b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.478+0000 7fcf14e4b640 1 -- 172.21.15.132:0/1596915602 >> 172.21.15.132:0/1596915602 conn(0x7fcf100b3e20 msgr2=0x7fcf100b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.479+0000 7fcf14e4b640 1 -- 172.21.15.132:0/1596915602 shutdown_connections 2024-03-13T07:34:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.479+0000 7fcf14e4b640 1 -- 172.21.15.132:0/1596915602 wait complete. 2024-03-13T07:34:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.480+0000 7f3e3ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.480+0000 7f3e3ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.480+0000 7f3e3ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.481+0000 7f3e3d59a640 1 -- 172.21.15.132:0/90752748 <== mon.0 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954498322 0 0) 0x7f3e40146a10 con 0x7f3e4006e610 2024-03-13T07:34:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.481+0000 7f3e3d59a640 10 monclient(hunting): _negotiate 2024-03-13T07:34:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.481+0000 7f3e3d59a640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.481+0000 7f3e3d59a640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.481+0000 7f3e3d59a640 10 monclient(hunting): my global_id is 4302 2024-03-13T07:34:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.481+0000 7f3e3d59a640 1 -- 172.21.15.132:0/90752748 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e28003520 con 0x7f3e4006e610 2024-03-13T07:34:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.480+0000 7fcf14e4b640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.480+0000 7fcf14e4b640 10 monclient: monmap: 2024-03-13T07:34:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:54.483 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.480+0000 7fcf14e4b640 1 librados: starting msgr at 2024-03-13T07:34:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.480+0000 7fcf14e4b640 1 librados: starting objecter 2024-03-13T07:34:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.481+0000 7fcf14e4b640 1 Processor -- start 2024-03-13T07:34:54.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.481+0000 7fcf14e4b640 1 -- start start 2024-03-13T07:34:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.481+0000 7fcf14e4b640 1 librados: setting wanted keys 2024-03-13T07:34:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.481+0000 7fcf14e4b640 1 librados: calling monclient init 2024-03-13T07:34:54.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.481+0000 7fcf14e4b640 10 monclient: init 2024-03-13T07:34:54.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.481+0000 7fcf14e4b640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.481+0000 7fcf14e4b640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T07:34:54.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.481+0000 7fcf14e4b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf1006ead0 0x7fcf10136d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.481+0000 7fcf14e4b640 10 monclient(hunting): picked mon.noname-a con 0x7fcf1006ead0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.481+0000 7fcf14e4b640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf10071d30 0x7fcf101db4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.482+0000 7fcf14e4b640 10 monclient(hunting): picked mon.noname-c con 0x7fcf10071d30 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.482+0000 7fcf14e4b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf1013bbb0 0x7fcf100b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.482+0000 7fcf14e4b640 10 monclient(hunting): picked mon.noname-b con 0x7fcf1013bbb0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.482+0000 7fcf14e4b640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.482+0000 7fcf14e4b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf10071000 con 0x7fcf1006ead0 2024-03-13T07:34:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.482+0000 7fcf14e4b640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.482+0000 7fcf14e4b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1013c020 con 0x7fcf1013bbb0 2024-03-13T07:34:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.482+0000 7fcf14e4b640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.482+0000 7fcf14e4b640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf10072160 con 0x7fcf10071d30 2024-03-13T07:34:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.482+0000 7fcf14e4b640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.482+0000 7fcf14e4b640 10 monclient(hunting): authenticate will time out at 1321.891140s 2024-03-13T07:34:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.483+0000 7fcf0dd9b640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf10071d30 0x7fcf101db4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.484+0000 7fcf0dd9b640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf10071d30 0x7fcf101db4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:42776/0 (socket says 172.21.15.132:42776) 2024-03-13T07:34:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.484+0000 7fcf0dd9b640 1 -- 172.21.15.132:0/883543987 learned_addr learned my addr 172.21.15.132:0/883543987 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.484+0000 7fcf0dd9b640 10 monclient(hunting): get_auth_request con 0x7fcf10071d30 auth_method 0 2024-03-13T07:34:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.484+0000 7fcf0dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.484+0000 7fcf0dd9b640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.484+0000 7fcf0dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.485+0000 7fcf0dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.485+0000 7fcf0dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.485+0000 7fcf0dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.486+0000 7fcf0ed9d640 1 --2- 172.21.15.132:0/883543987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf1013bbb0 0x7fcf100b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.486+0000 7fcf0ed9d640 10 monclient(hunting): get_auth_request con 0x7fcf1013bbb0 auth_method 0 2024-03-13T07:34:54.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.486+0000 7fcf0ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.486+0000 7fcf0ed9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.486+0000 7fcf0ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.486+0000 7fcf0ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.486+0000 7fcf0ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.486+0000 7fcf0ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.489+0000 7fcf0ed9d640 10 monclient(hunting): handle_auth_done global_id 4309 payload 824 2024-03-13T07:34:54.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.489+0000 7fcf0ed9d640 1 -- 172.21.15.132:0/883543987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf10071d30 msgr2=0x7fcf101db4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.489+0000 7fcf0ed9d640 1 --2- 172.21.15.132:0/883543987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf10071d30 0x7fcf101db4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.489+0000 7fcf0ed9d640 1 -- 172.21.15.132:0/883543987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf1006ead0 msgr2=0x7fcf10136d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.489+0000 7fcf0ed9d640 1 --2- 172.21.15.132:0/883543987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf1006ead0 0x7fcf10136d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.489+0000 7fcf0ed9d640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.489+0000 7fcf0ed9d640 1 monclient: found mon.noname-b 2024-03-13T07:34:54.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.489+0000 7fcf0ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.489+0000 7fcf0ed9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.489+0000 7fcf0ed9d640 1 -- 172.21.15.132:0/883543987 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf101dace0 con 0x7fcf1013bbb0 2024-03-13T07:34:54.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.499+0000 7fb8959bb640 10 client.4298.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-03-13T07:34:54.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.499+0000 7fb8959bb640 15 client.4298.objecter _session_command_op_assign -1 2 2024-03-13T07:34:54.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.499+0000 7fb8959bb640 20 client.4298.objecter _get_session s=0x7fb874073320 osd=1 3 2024-03-13T07:34:54.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.499+0000 7fb8959bb640 20 client.4298.objecter put_session s=0x7fb874073320 osd=1 3 2024-03-13T07:34:54.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.499+0000 7fb8959bb640 20 client.4298.objecter _get_session s=0x7fb874073320 osd=1 3 2024-03-13T07:34:54.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.499+0000 7fb8959bb640 15 client.4298.objecter _session_command_op_remove -1 2 2024-03-13T07:34:54.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.499+0000 7fb8959bb640 20 client.4298.objecter get_session s=0x7fb874073320 osd=1 3 2024-03-13T07:34:54.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.499+0000 7fb8959bb640 15 client.4298.objecter _session_command_op_assign 1 2 2024-03-13T07:34:54.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.499+0000 7fb8959bb640 20 client.4298.objecter put_session s=0x7fb874073320 osd=1 4 2024-03-13T07:34:54.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.499+0000 7fb8959bb640 10 client.4298.objecter _send_command 2 2024-03-13T07:34:54.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.499+0000 7fb8959bb640 1 -- 172.21.15.132:0/2155661044 --> [v2:172.21.15.132:6802/1071068206,v1:172.21.15.132:6803/1071068206] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb8880c2010 con 0x7fb8740739f0 2024-03-13T07:34:54.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.502+0000 7fcf0ed9d640 1 --2- 172.21.15.132:0/883543987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf1013bbb0 0x7fcf100b2e20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcf00034000 tx=0x7fcf000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.502+0000 7fcefeffd640 10 client.?.objecter ms_handle_connect 0x7fcf1013bbb0 2024-03-13T07:34:54.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.502+0000 7fcefeffd640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:54.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.506+0000 7f2d50ec2640 10 monclient: get_monmap_and_config 2024-03-13T07:34:54.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.506+0000 7f2d50ec2640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.506+0000 7f2d50ec2640 10 monclient: monmap: 2024-03-13T07:34:54.507 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.507 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.507 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:54.508321+0000 2024-03-13T07:34:54.507 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:54.508321+0000 2024-03-13T07:34:54.507 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.507 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.507 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:54.508 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:54.508 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:54.508 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:54.508 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:54.508 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.507+0000 7f2d50ec2640 1 Processor -- start 2024-03-13T07:34:54.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.507+0000 7f2d50ec2640 1 -- start start 2024-03-13T07:34:54.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.507+0000 7f2d50ec2640 10 monclient: init 2024-03-13T07:34:54.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.508+0000 7f3e3ed9d640 10 monclient(hunting): handle_auth_done global_id 4299 payload 274 2024-03-13T07:34:54.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.508+0000 7f3e3ed9d640 1 -- 172.21.15.132:0/90752748 >> v1:172.21.15.195:6789/0 conn(0x7f3e4006f950 legacy=0x7f3e40146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.508+0000 7f3e3ed9d640 1 -- 172.21.15.132:0/90752748 >> v1:172.21.15.132:6789/0 conn(0x7f3e4006e610 legacy=0x7f3e40143c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.508+0000 7f3e3ed9d640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.508+0000 7f3e3ed9d640 1 monclient: found mon.noname-a 2024-03-13T07:34:54.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.508+0000 7f3e3ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.508+0000 7f3e3ed9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.508+0000 7f3e3ed9d640 1 -- 172.21.15.132:0/90752748 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e40070fd0 con 0x7f3e40144340 2024-03-13T07:34:54.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.509+0000 7f2d50ec2640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.509+0000 7f2d50ec2640 10 monclient: _add_conns ranks=[4,0,3,1,2] 2024-03-13T07:34:54.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.509+0000 7f2d50ec2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2d4c0b5320 0x7f2d4c0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.509+0000 7f2d50ec2640 10 monclient(hunting): picked mon.noname-c con 0x7f2d4c0b5320 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:54.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.509+0000 7f2d50ec2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2d4c0b1ac0 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:54.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.509+0000 7f2d50ec2640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2d4c13fd40 0x7f2d4c0b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.509+0000 7f2d50ec2640 10 monclient(hunting): picked mon.noname-a con 0x7f2d4c13fd40 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:54.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.509+0000 7f2d50ec2640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4c0b1150 con 0x7f2d4c0b1ac0 2024-03-13T07:34:54.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.509+0000 7f2d50ec2640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.509+0000 7f2d50ec2640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d4c0b13c0 con 0x7f2d4c13fd40 2024-03-13T07:34:54.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.509+0000 7f2d50ec2640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.509+0000 7f2d50ec2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d4c0b16a0 con 0x7f2d4c0b5320 2024-03-13T07:34:54.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.509+0000 7f2d50ec2640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.509+0000 7f2d50ec2640 10 monclient(hunting): authenticate will time out at 1321.918575s 2024-03-13T07:34:54.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.517+0000 7fb88d7fa640 1 -- 172.21.15.132:0/2155661044 <== osd.1 v2:172.21.15.132:6802/1071068206 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb8880c2010 con 0x7fb8740739f0 2024-03-13T07:34:54.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.517+0000 7fb88d7fa640 10 client.4298.objecter ms_dispatch 0x7fb890000b90 command_reply(tid 2: 0 ) v1 2024-03-13T07:34:54.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.517+0000 7fb88d7fa640 10 client.4298.objecter _finish_command 2 = system:0 2024-03-13T07:34:54.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.517+0000 7fb88d7fa640 20 client.4298.objecter put_session s=0x7fb874073320 osd=1 4 2024-03-13T07:34:54.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.517+0000 7fb88d7fa640 15 client.4298.objecter _session_command_op_remove 1 2 2024-03-13T07:34:54.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.517+0000 7fb8959bb640 10 librados: watch_flush enter 2024-03-13T07:34:54.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.517+0000 7fb8959bb640 10 librados: watch_flush exit 2024-03-13T07:34:54.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.517+0000 7fb8959bb640 10 client.4298.objecter close_session for osd.1 2024-03-13T07:34:54.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.518+0000 7fb8959bb640 1 -- 172.21.15.132:0/2155661044 >> [v2:172.21.15.132:6802/1071068206,v1:172.21.15.132:6803/1071068206] conn(0x7fb8740739f0 msgr2=0x7fb874075e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.518+0000 7fb8959bb640 1 --2- 172.21.15.132:0/2155661044 >> [v2:172.21.15.132:6802/1071068206,v1:172.21.15.132:6803/1071068206] conn(0x7fb8740739f0 0x7fb874075e10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.518+0000 7fb8959bb640 20 client.4298.objecter put_session s=0x7fb874073320 osd=1 1 2024-03-13T07:34:54.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.518+0000 7fb8959bb640 20 client.4298.objecter shutdown clearing up homeless session... 2024-03-13T07:34:54.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.518+0000 7fb8959bb640 10 client.4298.objecter successfully canceled tick 2024-03-13T07:34:54.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.518+0000 7fb8959bb640 1 -- 172.21.15.132:0/2155661044 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb87406d5b0 msgr2=0x7fb87406fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.518+0000 7fb8959bb640 1 --2- 172.21.15.132:0/2155661044 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb87406d5b0 0x7fb87406fa70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb884006c90 tx=0x7fb884006c20 comp rx=0 tx=0).stop 2024-03-13T07:34:54.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.518+0000 7fb8959bb640 10 monclient: shutdown 2024-03-13T07:34:54.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.519+0000 7fb8959bb640 20 monclient: shutdown canceling and discarding version request 1 2024-03-13T07:34:54.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.519+0000 7fb8959bb640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.519+0000 7fb8959bb640 1 -- 172.21.15.132:0/2155661044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb88802ec20 msgr2=0x7fb88802c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.519+0000 7fb8959bb640 1 --2- 172.21.15.132:0/2155661044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb88802ec20 0x7fb88802c240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb87c02c5a0 tx=0x7fb87c000fe0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.519+0000 7fb8951ba640 1 -- 172.21.15.132:0/2155661044 reap_dead start 2024-03-13T07:34:54.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.520+0000 7fb8959bb640 1 -- 172.21.15.132:0/2155661044 shutdown_connections 2024-03-13T07:34:54.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.520+0000 7fb8959bb640 1 -- 172.21.15.132:0/2155661044 >> 172.21.15.132:0/2155661044 conn(0x7fb88803a740 msgr2=0x7fb888030370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.520+0000 7fb8959bb640 1 -- 172.21.15.132:0/2155661044 shutdown_connections 2024-03-13T07:34:54.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.520+0000 7fb8959bb640 1 -- 172.21.15.132:0/2155661044 wait complete. 2024-03-13T07:34:54.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.520+0000 7fb8959bb640 1 librados: shutdown 2024-03-13T07:34:54.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.526+0000 7f472edd8640 10 monclient: get_monmap_and_config 2024-03-13T07:34:54.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.526+0000 7f472edd8640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.526+0000 7f472edd8640 10 monclient: monmap: 2024-03-13T07:34:54.527 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.527 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.527 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:54.528429+0000 2024-03-13T07:34:54.527 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:54.528429+0000 2024-03-13T07:34:54.527 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.527 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.527 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:54.528 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:54.528 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:54.528 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:54.528 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:54.528 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.528 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.527+0000 7f472edd8640 1 Processor -- start 2024-03-13T07:34:54.528 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.527+0000 7f472edd8640 1 -- start start 2024-03-13T07:34:54.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.527+0000 7f472edd8640 10 monclient: init 2024-03-13T07:34:54.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.528+0000 7f472edd8640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.528+0000 7f472edd8640 10 monclient: _add_conns ranks=[4,0,2,1,3] 2024-03-13T07:34:54.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.528+0000 7f472edd8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f472806af30 0x7f47280adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.528+0000 7f472edd8640 10 monclient(hunting): picked mon.noname-c con 0x7f472806af30 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:54.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.528+0000 7f472edd8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f47280aea20 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:54.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.528+0000 7f472edd8640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4728148c20 0x7f472814b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.528+0000 7f472edd8640 10 monclient(hunting): picked mon.noname-b con 0x7f4728148c20 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.528+0000 7f472edd8640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f472813dc70 con 0x7f47280aea20 2024-03-13T07:34:54.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.528+0000 7f472edd8640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.528+0000 7f472edd8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472806b310 con 0x7f472806af30 2024-03-13T07:34:54.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.528+0000 7f472edd8640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.528+0000 7f472edd8640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47280afab0 con 0x7f4728148c20 2024-03-13T07:34:54.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.528+0000 7f472edd8640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.528+0000 7f472edd8640 10 monclient(hunting): authenticate will time out at 1321.938046s 2024-03-13T07:34:54.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.529+0000 7f472d375640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4728148c20 0x7f472814b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.530+0000 7f472d375640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4728148c20 0x7f472814b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:42790/0 (socket says 172.21.15.132:42790) 2024-03-13T07:34:54.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.530+0000 7f472d375640 1 -- 172.21.15.132:0/2794521861 learned_addr learned my addr 172.21.15.132:0/2794521861 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.530+0000 7f472d375640 10 monclient(hunting): get_auth_request con 0x7f4728148c20 auth_method 0 2024-03-13T07:34:54.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.530+0000 7f472d375640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.530+0000 7f472d375640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.530+0000 7f472d375640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.531+0000 7f472d375640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.531+0000 7f472d375640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.531+0000 7f472d375640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.534+0000 7f472d375640 10 monclient(hunting): handle_auth_done global_id 4304 payload 274 2024-03-13T07:34:54.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.534+0000 7f472d375640 1 -- 172.21.15.132:0/2794521861 >> v2:172.21.15.195:3300/0 conn(0x7f472806af30 msgr2=0x7f47280adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.534+0000 7f472d375640 1 --2- 172.21.15.132:0/2794521861 >> v2:172.21.15.195:3300/0 conn(0x7f472806af30 0x7f47280adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.534+0000 7f472d375640 1 -- 172.21.15.132:0/2794521861 >> v1:172.21.15.132:6789/0 conn(0x7f47280aea20 legacy=0x7f47280aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.534+0000 7f472d375640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.534+0000 7f472d375640 1 monclient: found mon.noname-b 2024-03-13T07:34:54.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.534+0000 7f472d375640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.534+0000 7f472d375640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.534+0000 7f472d375640 1 -- 172.21.15.132:0/2794521861 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47280b13d0 con 0x7f4728148c20 2024-03-13T07:34:54.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.537+0000 7f472d375640 1 --2- 172.21.15.132:0/2794521861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4728148c20 0x7f472814b010 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4720009e50 tx=0x7f472002bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7dae7cf89e8e9d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.539 INFO:teuthology.orchestra.run.smithi132.stdout:85899345929 2024-03-13T07:34:54.539 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-13T07:34:54.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f47277fe640 1 -- 172.21.15.132:0/2794521861 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472002ca80 con 0x7f4728148c20 2024-03-13T07:34:54.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f47277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f47277fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:54.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f47277fe640 10 monclient: dump: 2024-03-13T07:34:54.544 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.544 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.544 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.544 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.544 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.544 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.545 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.545 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.545 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.545 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f47277fe640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f47277fe640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f47277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f47277fe640 1 -- 172.21.15.132:0/2794521861 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472002cc20 con 0x7f4728148c20 2024-03-13T07:34:54.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f472edd8640 5 monclient: authenticate success, global_id 4304 2024-03-13T07:34:54.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f472edd8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:54.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f47277fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f47277fe640 1 -- 172.21.15.132:0/2794521861 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47200343f0 con 0x7f4728148c20 2024-03-13T07:34:54.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f47277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f47277fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:54.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f47277fe640 10 monclient: dump: 2024-03-13T07:34:54.545 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.546 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.546 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.546 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.546 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.546 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.546 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.546 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.546 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.546 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f472edd8640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:54.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.543+0000 7f472edd8640 4 set_mon_vals no callback set 2024-03-13T07:34:54.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.544+0000 7f472edd8640 10 monclient: shutdown 2024-03-13T07:34:54.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.544+0000 7f472edd8640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.544+0000 7f472edd8640 1 -- 172.21.15.132:0/2794521861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4728148c20 msgr2=0x7f472814b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.544+0000 7f472edd8640 1 --2- 172.21.15.132:0/2794521861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4728148c20 0x7f472814b010 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4720009e50 tx=0x7f472002bee0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.544+0000 7f472edd8640 1 -- 172.21.15.132:0/2794521861 shutdown_connections 2024-03-13T07:34:54.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.544+0000 7f472edd8640 1 --2- 172.21.15.132:0/2794521861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4728148c20 0x7f472814b010 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.544+0000 7f472edd8640 1 --2- 172.21.15.132:0/2794521861 >> v2:172.21.15.195:3300/0 conn(0x7f472806af30 0x7f47280adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.544+0000 7f472edd8640 1 -- 172.21.15.132:0/2794521861 >> 172.21.15.132:0/2794521861 conn(0x7f47280b4ed0 msgr2=0x7f47280b52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.544+0000 7f472edd8640 1 -- 172.21.15.132:0/2794521861 shutdown_connections 2024-03-13T07:34:54.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.544+0000 7f472edd8640 1 -- 172.21.15.132:0/2794521861 wait complete. 2024-03-13T07:34:54.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.545+0000 7f472edd8640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.545+0000 7f472edd8640 10 monclient: monmap: 2024-03-13T07:34:54.549 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.549 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:54.550 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:54.550 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:54.550 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.550 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.550 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:54.550 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:54.550 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:54.550 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.546+0000 7f472edd8640 1 librados: starting msgr at 2024-03-13T07:34:54.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.546+0000 7f472edd8640 1 librados: starting objecter 2024-03-13T07:34:54.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.546+0000 7f472edd8640 1 Processor -- start 2024-03-13T07:34:54.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.546+0000 7f472edd8640 1 -- start start 2024-03-13T07:34:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.546+0000 7f472edd8640 1 librados: setting wanted keys 2024-03-13T07:34:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.546+0000 7f472edd8640 1 librados: calling monclient init 2024-03-13T07:34:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.546+0000 7f472edd8640 10 monclient: init 2024-03-13T07:34:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.548+0000 7f472edd8640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.548+0000 7f472edd8640 10 monclient: _add_conns ranks=[1,2,0] 2024-03-13T07:34:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.548+0000 7f472edd8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f472806af30 0x7f4728071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.548+0000 7f472edd8640 10 monclient(hunting): picked mon.noname-b con 0x7f472806af30 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.548+0000 7f472edd8640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47280aea20 0x7f472814baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.548+0000 7f472edd8640 10 monclient(hunting): picked mon.noname-c con 0x7f47280aea20 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.549+0000 7f472edd8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4728148c20 0x7f47280ab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.549+0000 7f472edd8640 10 monclient(hunting): picked mon.noname-a con 0x7f4728148c20 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.549+0000 7f4727fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47280aea20 0x7f472814baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.549+0000 7f472edd8640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.549+0000 7f472edd8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47280ae500 con 0x7f4728148c20 2024-03-13T07:34:54.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.549+0000 7f472edd8640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.549+0000 7f472edd8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47280af6f0 con 0x7f472806af30 2024-03-13T07:34:54.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.549+0000 7f472edd8640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.549+0000 7f472edd8640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47280a9c50 con 0x7f47280aea20 2024-03-13T07:34:54.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.549+0000 7f472edd8640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.549+0000 7f472edd8640 10 monclient(hunting): authenticate will time out at 1321.959035s 2024-03-13T07:34:54.554 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.553+0000 7f4727fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47280aea20 0x7f472814baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:42796/0 (socket says 172.21.15.132:42796) 2024-03-13T07:34:54.554 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.553+0000 7f4727fff640 1 -- 172.21.15.132:0/1826027506 learned_addr learned my addr 172.21.15.132:0/1826027506 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.553+0000 7f4727fff640 10 monclient(hunting): get_auth_request con 0x7f47280aea20 auth_method 0 2024-03-13T07:34:54.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.553+0000 7f4727fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.553+0000 7f4727fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.553+0000 7f4727fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.554+0000 7f4727fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.554+0000 7f4727fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.554+0000 7f4727fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.557+0000 7fcefeffd640 1 -- 172.21.15.132:0/883543987 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0003a040 con 0x7fcf1013bbb0 2024-03-13T07:34:54.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.557+0000 7fcefeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.557+0000 7fcefeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:54.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.557+0000 7fcefeffd640 10 monclient: dump: 2024-03-13T07:34:54.559 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.559 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.559 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.559 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.559 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.559 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.559 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.559 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.559 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.559 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.557+0000 7fcefeffd640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.557+0000 7fcefeffd640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.557+0000 7fcefeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.557+0000 7fcf14e4b640 5 monclient: authenticate success, global_id 4309 2024-03-13T07:34:54.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.557+0000 7fcf14e4b640 10 monclient: _renew_subs 2024-03-13T07:34:54.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.557+0000 7fcf14e4b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.557+0000 7fcf14e4b640 1 -- 172.21.15.132:0/883543987 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf101db9e0 con 0x7fcf1013bbb0 2024-03-13T07:34:54.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.557+0000 7fcf14e4b640 10 client.4309.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.557+0000 7fcf14e4b640 10 monclient: _renew_subs 2024-03-13T07:34:54.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.557+0000 7fcf14e4b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.557+0000 7fcf14e4b640 1 -- 172.21.15.132:0/883543987 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf100b28b0 con 0x7fcf1013bbb0 2024-03-13T07:34:54.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.557+0000 7fcf14e4b640 1 librados: init done 2024-03-13T07:34:54.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.558+0000 7fcefeffd640 1 -- 172.21.15.132:0/883543987 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf00043070 con 0x7fcf1013bbb0 2024-03-13T07:34:54.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.558+0000 7fcefeffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.559+0000 7f2d4a59c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2d4c0b5320 0x7f2d4c0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.558+0000 7fcefffff640 4 set_mon_vals no callback set 2024-03-13T07:34:54.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.559+0000 7fcf14e4b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.559+0000 7fcefeffd640 1 -- 172.21.15.132:0/883543987 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00033b50 con 0x7fcf1013bbb0 2024-03-13T07:34:54.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.559+0000 7fcefeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.559+0000 7fcefeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:54.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.559+0000 7fcefeffd640 10 monclient: dump: 2024-03-13T07:34:54.562 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.562 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.562 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.562 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.562 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.562 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.562 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.562 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.563 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.563 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.559+0000 7fcf14e4b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.559+0000 7fcf14e4b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.559+0000 7fcf14e4b640 1 -- 172.21.15.132:0/883543987 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef4005080 con 0x7fcf1013bbb0 2024-03-13T07:34:54.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.559+0000 7f472cb74640 1 --2- 172.21.15.132:0/1826027506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f472806af30 0x7f4728071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.559+0000 7f2d4a59c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2d4c0b5320 0x7f2d4c0b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:48920/0 (socket says 172.21.15.132:48920) 2024-03-13T07:34:54.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.559+0000 7f2d4a59c640 1 -- 172.21.15.132:0/4253934313 learned_addr learned my addr 172.21.15.132:0/4253934313 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.559+0000 7f2d4a59c640 10 monclient(hunting): get_auth_request con 0x7f2d4c0b5320 auth_method 0 2024-03-13T07:34:54.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.559+0000 7f2d4a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.559+0000 7f2d4a59c640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.559+0000 7f2d4a59c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.560+0000 7f472cb74640 10 monclient(hunting): get_auth_request con 0x7f472806af30 auth_method 0 2024-03-13T07:34:54.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.560+0000 7f472cb74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.560+0000 7f472cb74640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.560+0000 7f472cb74640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.560+0000 7f472cb74640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.560+0000 7f472cb74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.561+0000 7f472cb74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.561+0000 7f2d4a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.561+0000 7f2d4a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.561+0000 7f2d4a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.566 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.562+0000 7f4727fff640 10 monclient(hunting): handle_auth_done global_id 4307 payload 824 2024-03-13T07:34:54.566 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.562+0000 7f4727fff640 1 -- 172.21.15.132:0/1826027506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f472806af30 msgr2=0x7f4728071990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.566 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.562+0000 7f4727fff640 1 --2- 172.21.15.132:0/1826027506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f472806af30 0x7f4728071990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.566 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.562+0000 7f4727fff640 1 -- 172.21.15.132:0/1826027506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4728148c20 msgr2=0x7f47280ab080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.566 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.562+0000 7f4727fff640 1 --2- 172.21.15.132:0/1826027506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4728148c20 0x7f47280ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.566 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.562+0000 7f4727fff640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.566 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.562+0000 7f4727fff640 1 monclient: found mon.noname-c 2024-03-13T07:34:54.567 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.562+0000 7f4727fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.567 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.562+0000 7f4727fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.567 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.562+0000 7f4727fff640 1 -- 172.21.15.132:0/1826027506 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47280aa6f0 con 0x7f47280aea20 2024-03-13T07:34:54.567 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.564+0000 7f4727fff640 1 --2- 172.21.15.132:0/1826027506 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47280aea20 0x7f472814baf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f471400c440 tx=0x7f471400c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.567 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.565+0000 7f47257fa640 10 client.?.objecter ms_handle_connect 0x7f47280aea20 2024-03-13T07:34:54.567 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.565+0000 7f47257fa640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:54.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.567+0000 7f2d4a59c640 10 monclient(hunting): handle_auth_done global_id 4315 payload 274 2024-03-13T07:34:54.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.567+0000 7f2d4a59c640 1 -- 172.21.15.132:0/4253934313 >> v2:172.21.15.132:3300/0 conn(0x7f2d4c13fd40 msgr2=0x7f2d4c0b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.567+0000 7f2d4a59c640 1 --2- 172.21.15.132:0/4253934313 >> v2:172.21.15.132:3300/0 conn(0x7f2d4c13fd40 0x7f2d4c0b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.567+0000 7f2d4a59c640 1 -- 172.21.15.132:0/4253934313 >> v1:172.21.15.132:6789/0 conn(0x7f2d4c0b1ac0 legacy=0x7f2d4c13f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.567+0000 7f2d4a59c640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.567+0000 7f2d4a59c640 1 monclient: found mon.noname-c 2024-03-13T07:34:54.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.567+0000 7f2d4a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.567+0000 7f2d4a59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.567+0000 7f2d4a59c640 1 -- 172.21.15.132:0/4253934313 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c0afa20 con 0x7f2d4c0b5320 2024-03-13T07:34:54.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.570+0000 7f47257fa640 1 -- 172.21.15.132:0/1826027506 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714017070 con 0x7f47280aea20 2024-03-13T07:34:54.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.570+0000 7f47257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.571+0000 7f47257fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:54.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.571+0000 7f47257fa640 10 monclient: dump: 2024-03-13T07:34:54.572 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.572 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.572 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.572 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.572 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.572 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.572 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.572 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.572 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.573 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.571+0000 7f47257fa640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.571+0000 7f47257fa640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.571+0000 7f47257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.571+0000 7f47257fa640 1 -- 172.21.15.132:0/1826027506 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47140035f0 con 0x7f47280aea20 2024-03-13T07:34:54.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.571+0000 7f472edd8640 5 monclient: authenticate success, global_id 4307 2024-03-13T07:34:54.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.571+0000 7f472edd8640 10 monclient: _renew_subs 2024-03-13T07:34:54.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.571+0000 7f472edd8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.571+0000 7f472edd8640 1 -- 172.21.15.132:0/1826027506 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47280aac60 con 0x7f47280aea20 2024-03-13T07:34:54.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.571+0000 7f47257fa640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.571+0000 7f472edd8640 10 client.4307.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.571+0000 7f4726ffd640 4 set_mon_vals no callback set 2024-03-13T07:34:54.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.572+0000 7f47257fa640 1 -- 172.21.15.132:0/1826027506 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140039e0 con 0x7f47280aea20 2024-03-13T07:34:54.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.572+0000 7f47257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.572+0000 7f47257fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:54.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.572+0000 7f47257fa640 10 monclient: dump: 2024-03-13T07:34:54.574 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.574 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.574 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.574 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.574 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.575 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.575 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.575 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.575 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.575 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.572+0000 7f472edd8640 10 monclient: _renew_subs 2024-03-13T07:34:54.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.572+0000 7f472edd8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.572+0000 7f472edd8640 1 -- 172.21.15.132:0/1826027506 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472814c030 con 0x7f47280aea20 2024-03-13T07:34:54.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.572+0000 7f472edd8640 1 librados: init done 2024-03-13T07:34:54.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.573+0000 7f472edd8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.573+0000 7f472edd8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.573+0000 7f472edd8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:54.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.573+0000 7f472edd8640 1 -- 172.21.15.132:0/1826027506 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472804f000 con 0x7f47280aea20 2024-03-13T07:34:54.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.579+0000 7f42bc521640 10 monclient: get_monmap_and_config 2024-03-13T07:34:54.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.579+0000 7f42bc521640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.579+0000 7f42bc521640 10 monclient: monmap: 2024-03-13T07:34:54.579 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.579 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.579 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:54.580810+0000 2024-03-13T07:34:54.580 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:54.580810+0000 2024-03-13T07:34:54.580 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.580 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.580 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:54.580 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:54.580 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:54.580 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:54.580 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:54.580 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.579+0000 7f42bc521640 1 Processor -- start 2024-03-13T07:34:54.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.579+0000 7f42bc521640 1 -- start start 2024-03-13T07:34:54.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.579+0000 7f42bc521640 10 monclient: init 2024-03-13T07:34:54.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.580+0000 7f42bc521640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.580+0000 7f42bc521640 10 monclient: _add_conns ranks=[1,4,3,0,2] 2024-03-13T07:34:54.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.580+0000 7f42bc521640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f42b40b5320 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:54.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.580+0000 7f42bc521640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f42b40b0220 0x7f42b413f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.580+0000 7f42bc521640 10 monclient(hunting): picked mon.noname-c con 0x7f42b40b0220 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:54.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.580+0000 7f42bc521640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f42b413fb70 0x7f42b414cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.580+0000 7f42bc521640 10 monclient(hunting): picked mon.noname-a con 0x7f42b413fb70 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:54.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.581+0000 7f42bc521640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b40b1150 con 0x7f42b40b5320 2024-03-13T07:34:54.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.581+0000 7f42bc521640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.581+0000 7f42bc521640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b40b0690 con 0x7f42b413fb70 2024-03-13T07:34:54.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.581+0000 7f42bc521640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.581+0000 7f42bc521640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b413ffa0 con 0x7f42b40b0220 2024-03-13T07:34:54.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.581+0000 7f42bc521640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.581+0000 7f42bc521640 10 monclient(hunting): authenticate will time out at 1321.990191s 2024-03-13T07:34:54.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.582+0000 7f3e3ed9d640 1 --2- 172.21.15.132:0/90752748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e40144340 0x7f3e40152290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3e340097a0 tx=0x7f3e3402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b6f0623be437cc server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.589 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.588+0000 7f42baabe640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f42b413fb70 0x7f42b414cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.593+0000 7f42baabe640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f42b413fb70 0x7f42b414cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56712/0 (socket says 172.21.15.132:56712) 2024-03-13T07:34:54.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.593+0000 7f42baabe640 1 -- 172.21.15.132:0/1989778500 learned_addr learned my addr 172.21.15.132:0/1989778500 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.593+0000 7f42baabe640 10 monclient(hunting): get_auth_request con 0x7f42b413fb70 auth_method 0 2024-03-13T07:34:54.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.593+0000 7f42baabe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.593+0000 7f42baabe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.593+0000 7f42baabe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.595+0000 7f42baabe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.595+0000 7f42baabe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.595+0000 7f42baabe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.595+0000 7f3e3d59a640 1 -- 172.21.15.132:0/90752748 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e340315a0 con 0x7f3e40144340 2024-03-13T07:34:54.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.595+0000 7f3e3d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.595+0000 7f3e3d59a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:54.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.595+0000 7f3e3d59a640 10 monclient: dump: 2024-03-13T07:34:54.598 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.598 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.598 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.598 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.598 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.598 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.598 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.598 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.598 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.598 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.595+0000 7f3e3d59a640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.595+0000 7f3e3d59a640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.595+0000 7f3e3d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.595+0000 7f3e44ec1640 5 monclient: authenticate success, global_id 4299 2024-03-13T07:34:54.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.595+0000 7f3e44ec1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:54.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.595+0000 7f3e3d59a640 1 -- 172.21.15.132:0/90752748 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e34037070 con 0x7f3e40144340 2024-03-13T07:34:54.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.595+0000 7f3e3d59a640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.595+0000 7f3e44ec1640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:54.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.595+0000 7f3e44ec1640 4 set_mon_vals no callback set 2024-03-13T07:34:54.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.596+0000 7f3e3d59a640 1 -- 172.21.15.132:0/90752748 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34031960 con 0x7f3e40144340 2024-03-13T07:34:54.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.596+0000 7f3e3d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.596+0000 7f3e3d59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:54.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.596+0000 7f3e3d59a640 10 monclient: dump: 2024-03-13T07:34:54.599 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.599 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.600 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.600 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.600 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.600 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.600 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.600 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.600 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.600 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.596+0000 7f3e44ec1640 10 monclient: shutdown 2024-03-13T07:34:54.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.596+0000 7f3e44ec1640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.596+0000 7f3e44ec1640 1 -- 172.21.15.132:0/90752748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e40144340 msgr2=0x7f3e40152290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.596+0000 7f3e44ec1640 1 --2- 172.21.15.132:0/90752748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e40144340 0x7f3e40152290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3e340097a0 tx=0x7f3e3402b840 comp rx=0 tx=0).stop 2024-03-13T07:34:54.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.597+0000 7f3e44ec1640 1 -- 172.21.15.132:0/90752748 shutdown_connections 2024-03-13T07:34:54.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.597+0000 7f3e44ec1640 1 --2- 172.21.15.132:0/90752748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e40144340 0x7f3e40152290 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.597+0000 7f3e44ec1640 1 -- 172.21.15.132:0/90752748 >> 172.21.15.132:0/90752748 conn(0x7f3e400b3e20 msgr2=0x7f3e400b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.597+0000 7f3e44ec1640 1 -- 172.21.15.132:0/90752748 shutdown_connections 2024-03-13T07:34:54.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.597+0000 7f3e44ec1640 1 -- 172.21.15.132:0/90752748 wait complete. 2024-03-13T07:34:54.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.598+0000 7f42baabe640 10 monclient(hunting): handle_auth_done global_id 4305 payload 274 2024-03-13T07:34:54.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.598+0000 7f42baabe640 1 -- 172.21.15.132:0/1989778500 >> v2:172.21.15.195:3300/0 conn(0x7f42b40b0220 msgr2=0x7f42b413f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.598+0000 7f42baabe640 1 --2- 172.21.15.132:0/1989778500 >> v2:172.21.15.195:3300/0 conn(0x7f42b40b0220 0x7f42b413f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.598+0000 7f42baabe640 1 -- 172.21.15.132:0/1989778500 >> v1:172.21.15.195:6789/0 conn(0x7f42b40b5320 legacy=0x7f42b40b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.598+0000 7f42baabe640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.598+0000 7f42baabe640 1 monclient: found mon.noname-a 2024-03-13T07:34:54.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.598+0000 7f42baabe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.598+0000 7f42baabe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.598+0000 7f42baabe640 1 -- 172.21.15.132:0/1989778500 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b40b13c0 con 0x7f42b413fb70 2024-03-13T07:34:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.598+0000 7f3e44ec1640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.598+0000 7f3e44ec1640 10 monclient: monmap: 2024-03-13T07:34:54.605 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.606 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:54.606 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:54.606 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:54.607 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.598+0000 7f3e44ec1640 1 librados: starting msgr at 2024-03-13T07:34:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.598+0000 7f3e44ec1640 1 librados: starting objecter 2024-03-13T07:34:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.598+0000 7f3e44ec1640 1 Processor -- start 2024-03-13T07:34:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.599+0000 7f3e44ec1640 1 -- start start 2024-03-13T07:34:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.599+0000 7f3e44ec1640 1 librados: setting wanted keys 2024-03-13T07:34:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.599+0000 7f3e44ec1640 1 librados: calling monclient init 2024-03-13T07:34:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.599+0000 7f3e44ec1640 10 monclient: init 2024-03-13T07:34:54.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.599+0000 7f3e44ec1640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.599+0000 7f3e44ec1640 10 monclient: _add_conns ranks=[1,0,2] 2024-03-13T07:34:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.599+0000 7f3e44ec1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e4006e610 0x7f3e400b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.599+0000 7f3e44ec1640 10 monclient(hunting): picked mon.noname-b con 0x7f3e4006e610 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.599+0000 7f3e44ec1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e4006f950 0x7f3e40147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.600+0000 7f3e44ec1640 10 monclient(hunting): picked mon.noname-a con 0x7f3e4006f950 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.600+0000 7f3e44ec1640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e40144340 0x7f3e40148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.600+0000 7f3e44ec1640 10 monclient(hunting): picked mon.noname-c con 0x7f3e40144340 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.600+0000 7f3e44ec1640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.600+0000 7f3e44ec1640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4006e9f0 con 0x7f3e4006f950 2024-03-13T07:34:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.600+0000 7f3e44ec1640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.600+0000 7f3e44ec1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e401478b0 con 0x7f3e4006e610 2024-03-13T07:34:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.600+0000 7f3e44ec1640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.600+0000 7f3e44ec1640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4006eb70 con 0x7f3e40144340 2024-03-13T07:34:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.600+0000 7f3e44ec1640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.600+0000 7f3e44ec1640 10 monclient(hunting): authenticate will time out at 1322.009448s 2024-03-13T07:34:54.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.602+0000 7f8906935640 10 monclient: get_monmap_and_config 2024-03-13T07:34:54.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.602+0000 7f8906935640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.602+0000 7f8906935640 10 monclient: monmap: 2024-03-13T07:34:54.610 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.610 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.610 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:54.604681+0000 2024-03-13T07:34:54.610 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:54.604681+0000 2024-03-13T07:34:54.610 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.610 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.610 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:54.610 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:54.610 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:54.610 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:54.610 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:54.610 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.603+0000 7f8906935640 1 Processor -- start 2024-03-13T07:34:54.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.603+0000 7f8906935640 1 -- start start 2024-03-13T07:34:54.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.603+0000 7f42baabe640 1 --2- 172.21.15.132:0/1989778500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42b413fb70 0x7f42b414cef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f42b40b0810 tx=0x7f42b002b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fad7ffaf0bdb1a server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.603+0000 7f8906935640 10 monclient: init 2024-03-13T07:34:54.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.604+0000 7f8906935640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.604+0000 7f8906935640 10 monclient: _add_conns ranks=[0,4,1,2,3] 2024-03-13T07:34:54.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.604+0000 7f8906935640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f89000b5320 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:54.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.604+0000 7f8906935640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f89000b0220 0x7f890013f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.604+0000 7f8906935640 10 monclient(hunting): picked mon.noname-c con 0x7f89000b0220 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:54.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.604+0000 7f8906935640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f890013fb70 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:54.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.604+0000 7f8906935640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89000b1150 con 0x7f89000b5320 2024-03-13T07:34:54.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.604+0000 7f8906935640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f890013ffa0 con 0x7f890013fb70 2024-03-13T07:34:54.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.604+0000 7f8906935640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.604+0000 7f8906935640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89000b0690 con 0x7f89000b0220 2024-03-13T07:34:54.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.604+0000 7f8906935640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.604+0000 7f8906935640 10 monclient(hunting): authenticate will time out at 1322.014014s 2024-03-13T07:34:54.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.607+0000 7f3e3dd9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e4006f950 0x7f3e40147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.608+0000 7f88fffff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f89000b5320 0x7f89000b1ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:36078/0 (socket says 172.21.15.132:36078) 2024-03-13T07:34:54.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.608+0000 7f88fffff640 1 -- 172.21.15.132:0/4056211493 learned_addr learned my addr 172.21.15.132:0/4056211493 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:34:54.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.608+0000 7f3e3dd9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e4006f950 0x7f3e40147d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56728/0 (socket says 172.21.15.132:56728) 2024-03-13T07:34:54.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.608+0000 7f3e3dd9b640 1 -- 172.21.15.132:0/1796523488 learned_addr learned my addr 172.21.15.132:0/1796523488 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.608+0000 7f3e3dd9b640 10 monclient(hunting): get_auth_request con 0x7f3e4006f950 auth_method 0 2024-03-13T07:34:54.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.608+0000 7f3e3dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.608+0000 7f3e3dd9b640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.608+0000 7f3e3dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.609+0000 7f42b92bb640 1 -- 172.21.15.132:0/1989778500 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b0031760 con 0x7f42b413fb70 2024-03-13T07:34:54.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.609+0000 7f42b92bb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.609+0000 7f42b92bb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:54.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.609+0000 7f42b92bb640 10 monclient: dump: 2024-03-13T07:34:54.616 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.616 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.616 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.616 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.616 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.616 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.616 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.616 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.616 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.616 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.609+0000 7f42b92bb640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.609+0000 7f42b92bb640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.609+0000 7f42b92bb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.609+0000 7f42b92bb640 1 -- 172.21.15.132:0/1989778500 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b0037070 con 0x7f42b413fb70 2024-03-13T07:34:54.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.609+0000 7f42b92bb640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.609+0000 7f42b92bb640 1 -- 172.21.15.132:0/1989778500 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b0031b60 con 0x7f42b413fb70 2024-03-13T07:34:54.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.609+0000 7f42b92bb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.609+0000 7f42b92bb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:54.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.609+0000 7f42b92bb640 10 monclient: dump: 2024-03-13T07:34:54.617 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.617 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.617 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.617 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.617 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.617 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.617 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.617 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.617 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.618 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.609+0000 7f42bc521640 5 monclient: authenticate success, global_id 4305 2024-03-13T07:34:54.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.609+0000 7f42bc521640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:54.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.609+0000 7f42bc521640 4 set_mon_vals no callback set 2024-03-13T07:34:54.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.610+0000 7f42bc521640 10 monclient: shutdown 2024-03-13T07:34:54.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.610+0000 7f42bc521640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.610+0000 7f42bc521640 1 -- 172.21.15.132:0/1989778500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42b413fb70 msgr2=0x7f42b414cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.610+0000 7f42bc521640 1 --2- 172.21.15.132:0/1989778500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42b413fb70 0x7f42b414cef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f42b40b0810 tx=0x7f42b002b800 comp rx=0 tx=0).stop 2024-03-13T07:34:54.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.610+0000 7f42bc521640 1 -- 172.21.15.132:0/1989778500 shutdown_connections 2024-03-13T07:34:54.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.610+0000 7f42bc521640 1 --2- 172.21.15.132:0/1989778500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42b413fb70 0x7f42b414cef0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.610+0000 7f42bc521640 1 --2- 172.21.15.132:0/1989778500 >> v2:172.21.15.195:3300/0 conn(0x7f42b40b0220 0x7f42b413f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.610+0000 7f42bc521640 1 -- 172.21.15.132:0/1989778500 >> 172.21.15.132:0/1989778500 conn(0x7f42b406d2b0 msgr2=0x7f42b406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.610+0000 7f3e3dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.610+0000 7f3e3dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.610+0000 7f3e3dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.611+0000 7f42bc521640 1 -- 172.21.15.132:0/1989778500 shutdown_connections 2024-03-13T07:34:54.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.611+0000 7f42bc521640 1 -- 172.21.15.132:0/1989778500 wait complete. 2024-03-13T07:34:54.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.611+0000 7f42bc521640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.611+0000 7f42bc521640 10 monclient: monmap: 2024-03-13T07:34:54.620 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.620 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:54.620 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:54.620 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:54.620 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.621 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.621 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:54.621 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:54.621 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:54.621 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.611+0000 7f05f9974640 10 monclient: get_monmap_and_config 2024-03-13T07:34:54.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.611+0000 7f05f9974640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.611+0000 7f05f9974640 10 monclient: monmap: 2024-03-13T07:34:54.621 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.621 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.621 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:54.613694+0000 2024-03-13T07:34:54.622 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:54.613694+0000 2024-03-13T07:34:54.622 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.622 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.622 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:54.622 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:54.622 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:54.622 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:54.622 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:54.622 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.611+0000 7f42bc521640 1 librados: starting msgr at 2024-03-13T07:34:54.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.611+0000 7f42bc521640 1 librados: starting objecter 2024-03-13T07:34:54.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 1 Processor -- start 2024-03-13T07:34:54.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 1 -- start start 2024-03-13T07:34:54.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 1 librados: setting wanted keys 2024-03-13T07:34:54.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 1 librados: calling monclient init 2024-03-13T07:34:54.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 10 monclient: init 2024-03-13T07:34:54.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 10 monclient: _add_conns ranks=[1,0,2] 2024-03-13T07:34:54.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42b40b0220 0x7f42b41cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 10 monclient(hunting): picked mon.noname-b con 0x7f42b40b0220 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42b40b5320 0x7f42b41d8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 10 monclient(hunting): picked mon.noname-a con 0x7f42b40b5320 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42b413fb70 0x7f42b41d8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 10 monclient(hunting): picked mon.noname-c con 0x7f42b413fb70 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b40b4800 con 0x7f42b40b5320 2024-03-13T07:34:54.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b40afd00 con 0x7f42b40b0220 2024-03-13T07:34:54.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b40b4980 con 0x7f42b413fb70 2024-03-13T07:34:54.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.612+0000 7f42bc521640 10 monclient(hunting): authenticate will time out at 1322.021726s 2024-03-13T07:34:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 1 Processor -- start 2024-03-13T07:34:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 1 -- start start 2024-03-13T07:34:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 10 monclient: init 2024-03-13T07:34:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 10 monclient: _add_conns ranks=[4,0,2,3,1] 2024-03-13T07:34:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f05f40b5320 0x7f05f40b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 10 monclient(hunting): picked mon.noname-c con 0x7f05f40b5320 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f05f40b1ac0 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05f413fd40 0x7f05f40b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 10 monclient(hunting): picked mon.noname-b con 0x7f05f413fd40 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f40b1150 con 0x7f05f40b1ac0 2024-03-13T07:34:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f40b13c0 con 0x7f05f40b5320 2024-03-13T07:34:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f40b16a0 con 0x7f05f413fd40 2024-03-13T07:34:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.613+0000 7f05f9974640 10 monclient(hunting): authenticate will time out at 1322.022760s 2024-03-13T07:34:54.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.614+0000 7f3e3dd9b640 10 monclient(hunting): handle_auth_done global_id 4308 payload 824 2024-03-13T07:34:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.614+0000 7f3e3dd9b640 1 -- 172.21.15.132:0/1796523488 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e40144340 msgr2=0x7f3e40148250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.614+0000 7f3e3dd9b640 1 --2- 172.21.15.132:0/1796523488 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e40144340 0x7f3e40148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.614+0000 7f3e3dd9b640 1 -- 172.21.15.132:0/1796523488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e4006e610 msgr2=0x7f3e400b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.614+0000 7f3e3dd9b640 1 --2- 172.21.15.132:0/1796523488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e4006e610 0x7f3e400b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.614+0000 7f3e3dd9b640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.614+0000 7f3e3dd9b640 1 monclient: found mon.noname-a 2024-03-13T07:34:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.614+0000 7f3e3dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.614+0000 7f3e3dd9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.614+0000 7f3e3dd9b640 1 -- 172.21.15.132:0/1796523488 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4014ab70 con 0x7f3e4006f950 2024-03-13T07:34:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.619+0000 7f3e3dd9b640 1 --2- 172.21.15.132:0/1796523488 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e4006f950 0x7f3e40147d10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000c450 tx=0x7f3e3000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.619+0000 7f3e2effd640 10 client.?.objecter ms_handle_connect 0x7f3e4006f950 2024-03-13T07:34:54.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.619+0000 7f3e2effd640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.620+0000 7f05ebfff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f05f40b1ac0 0x7f05f413f630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:36094/0 (socket says 172.21.15.132:36094) 2024-03-13T07:34:54.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.620+0000 7f05ebfff640 1 -- 172.21.15.132:0/114715847 learned_addr learned my addr 172.21.15.132:0/114715847 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:34:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.622+0000 7f88feffd640 1 -- 172.21.15.132:0/4056211493 <== mon.0 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886827190 0 0) 0x7f89000b1150 con 0x7f89000b5320 2024-03-13T07:34:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.622+0000 7f88feffd640 10 monclient(hunting): _negotiate 2024-03-13T07:34:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.622+0000 7f88feffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.622+0000 7f88feffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.622+0000 7f88feffd640 10 monclient(hunting): my global_id is 4311 2024-03-13T07:34:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.622+0000 7f88feffd640 1 -- 172.21.15.132:0/4056211493 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88e8003520 con 0x7f89000b5320 2024-03-13T07:34:54.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.625+0000 7f42b9abc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42b40b5320 0x7f42b41d8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.628+0000 7f05f27fc640 1 -- 172.21.15.132:0/114715847 <== mon.0 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55122086 0 0) 0x7f05f40b1150 con 0x7f05f40b1ac0 2024-03-13T07:34:54.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.628+0000 7f05f27fc640 10 monclient(hunting): _negotiate 2024-03-13T07:34:54.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.628+0000 7f05f27fc640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.628+0000 7f05f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.628+0000 7f05f27fc640 10 monclient(hunting): my global_id is 4314 2024-03-13T07:34:54.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.628+0000 7f05f27fc640 1 -- 172.21.15.132:0/114715847 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e0003520 con 0x7f05f40b1ac0 2024-03-13T07:34:54.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.628+0000 7f42b9abc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42b40b5320 0x7f42b41d8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56740/0 (socket says 172.21.15.132:56740) 2024-03-13T07:34:54.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.628+0000 7f42b9abc640 1 -- 172.21.15.132:0/547973600 learned_addr learned my addr 172.21.15.132:0/547973600 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.628+0000 7f42b9abc640 10 monclient(hunting): get_auth_request con 0x7f42b40b5320 auth_method 0 2024-03-13T07:34:54.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.628+0000 7f42b9abc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.628+0000 7f42b9abc640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.628+0000 7f42b9abc640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.628+0000 7f88feffd640 1 -- 172.21.15.132:0/4056211493 <== mon.0 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1300385243 0 0) 0x7f88e8003520 con 0x7f89000b5320 2024-03-13T07:34:54.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 1 -- 172.21.15.132:0/4056211493 >> v2:172.21.15.195:3300/0 conn(0x7f89000b0220 msgr2=0x7f890013f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 1 --2- 172.21.15.132:0/4056211493 >> v2:172.21.15.195:3300/0 conn(0x7f89000b0220 0x7f890013f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 1 -- 172.21.15.132:0/4056211493 >> v1:172.21.15.195:6789/0 conn(0x7f890013fb70 legacy=0x7f890014cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 1 monclient: found mon.noname-a-legacy 2024-03-13T07:34:54.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.132:6789/0 2024-03-13T07:34:54.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 1 -- 172.21.15.132:0/4056211493 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89000b13c0 con 0x7f89000b5320 2024-03-13T07:34:54.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 1 -- 172.21.15.132:0/4056211493 <== mon.0 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1639563336 0 0) 0x7f88f0001670 con 0x7f89000b5320 2024-03-13T07:34:54.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f8906935640 5 monclient: authenticate success, global_id 4311 2024-03-13T07:34:54.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f8906935640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:54.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-03-13T07:34:54.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient: dump: 2024-03-13T07:34:54.635 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.635 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.635 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.635 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.635 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.635 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.635 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.635 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.635 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.635 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] but i'm connected to v1:172.21.15.132:6789/0, reconnecting 2024-03-13T07:34:54.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 1 -- 172.21.15.132:0/4056211493 >> v1:172.21.15.132:6789/0 conn(0x7f89000b5320 legacy=0x7f89000b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T07:34:54.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 1 --2- 172.21.15.132:0/4056211493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88e8003a00 0x7f88e8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient(hunting): picked mon.a con 0x7f88e8003a00 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 1 --2- 172.21.15.132:0/4056211493 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88e8004800 0x7f88e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient(hunting): picked mon.c con 0x7f88e8004800 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 1 --2- 172.21.15.132:0/4056211493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88e8005410 0x7f88e800ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient(hunting): picked mon.b con 0x7f88e8005410 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 1 -- 172.21.15.132:0/4056211493 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89000b0690 con 0x7f88e8003a00 2024-03-13T07:34:54.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 1 -- 172.21.15.132:0/4056211493 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e800c350 con 0x7f88e8005410 2024-03-13T07:34:54.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 1 -- 172.21.15.132:0/4056211493 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e800c550 con 0x7f88e8004800 2024-03-13T07:34:54.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f88feffd640 10 monclient(hunting): _finish_auth 0 2024-03-13T07:34:54.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f8906935640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:54.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f42b9abc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f42b9abc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.629+0000 7f42b9abc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.633+0000 7f42b9abc640 10 monclient(hunting): handle_auth_done global_id 4317 payload 824 2024-03-13T07:34:54.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.633+0000 7f42b9abc640 1 -- 172.21.15.132:0/547973600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42b413fb70 msgr2=0x7f42b41d8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.633+0000 7f42b9abc640 1 --2- 172.21.15.132:0/547973600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42b413fb70 0x7f42b41d8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.633+0000 7f42b9abc640 1 -- 172.21.15.132:0/547973600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42b40b0220 msgr2=0x7f42b41cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.633+0000 7f42b9abc640 1 --2- 172.21.15.132:0/547973600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42b40b0220 0x7f42b41cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.633+0000 7f42b9abc640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.633+0000 7f42b9abc640 1 monclient: found mon.noname-a 2024-03-13T07:34:54.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.633+0000 7f42b9abc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.633+0000 7f42b9abc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.633+0000 7f42b9abc640 1 -- 172.21.15.132:0/547973600 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b41d7e40 con 0x7f42b40b5320 2024-03-13T07:34:54.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.635+0000 7f47257fa640 1 -- 172.21.15.132:0/1826027506 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f471401a070 con 0x7f47280aea20 2024-03-13T07:34:54.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.636+0000 7f47257fa640 1 --2- 172.21.15.132:0/1826027506 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f470c06d5b0 0x7f470c06fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.636+0000 7f472cb74640 1 --2- 172.21.15.132:0/1826027506 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f470c06d5b0 0x7f470c06fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.636+0000 7f472cb74640 10 monclient: get_auth_request con 0x7f470c06d5b0 auth_method 0 2024-03-13T07:34:54.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.636+0000 7f88fffff640 1 --2- 172.21.15.132:0/4056211493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88e8003a00 0x7f88e8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.636+0000 7f472cb74640 1 --2- 172.21.15.132:0/1826027506 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f470c06d5b0 0x7f470c06fa70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f471c009990 tx=0x7f471c009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.637+0000 7f47257fa640 10 client.4307.objecter ms_handle_connect 0x7f470c06d5b0 2024-03-13T07:34:54.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.640+0000 7f88fffff640 10 monclient(hunting): get_auth_request con 0x7f88e8003a00 auth_method 0 2024-03-13T07:34:54.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.640+0000 7f88fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.640+0000 7f88fffff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.640+0000 7f88fffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-03-13T07:34:54.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.641+0000 7f42b9abc640 1 --2- 172.21.15.132:0/547973600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42b40b5320 0x7f42b41d8300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00ba50 tx=0x7f42ac00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.641+0000 7f42aaffd640 10 client.?.objecter ms_handle_connect 0x7f42b40b5320 2024-03-13T07:34:54.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.641+0000 7f42aaffd640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:54.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.641+0000 7f05f27fc640 1 -- 172.21.15.132:0/114715847 <== mon.0 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2165148597 0 0) 0x7f05e0003520 con 0x7f05f40b1ac0 2024-03-13T07:34:54.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.641+0000 7f05f27fc640 1 -- 172.21.15.132:0/114715847 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05f413fd40 msgr2=0x7f05f40b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.641+0000 7f05f27fc640 1 --2- 172.21.15.132:0/114715847 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05f413fd40 0x7f05f40b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.641+0000 7f05f27fc640 1 -- 172.21.15.132:0/114715847 >> v2:172.21.15.195:3300/0 conn(0x7f05f40b5320 msgr2=0x7f05f40b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.641+0000 7f05f27fc640 1 --2- 172.21.15.132:0/114715847 >> v2:172.21.15.195:3300/0 conn(0x7f05f40b5320 0x7f05f40b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.641+0000 7f05f27fc640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.641+0000 7f05f27fc640 1 monclient: found mon.noname-a-legacy 2024-03-13T07:34:54.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.641+0000 7f05f27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.641+0000 7f05f27fc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.132:6789/0 2024-03-13T07:34:54.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.641+0000 7f05f27fc640 1 -- 172.21.15.132:0/114715847 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f40afa20 con 0x7f05f40b1ac0 2024-03-13T07:34:54.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 1 -- 172.21.15.132:0/114715847 <== mon.0 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1639563336 0 0) 0x7f05dc001670 con 0x7f05f40b1ac0 2024-03-13T07:34:54.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-03-13T07:34:54.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 10 monclient: dump: 2024-03-13T07:34:54.644 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.644 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.644 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.644 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.644 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.644 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.644 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.644 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.644 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.644 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] but i'm connected to v1:172.21.15.132:6789/0, reconnecting 2024-03-13T07:34:54.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 1 -- 172.21.15.132:0/114715847 >> v1:172.21.15.132:6789/0 conn(0x7f05f40b1ac0 legacy=0x7f05f413f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 10 monclient: _add_conns ranks=[2,1,0] 2024-03-13T07:34:54.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 1 --2- 172.21.15.132:0/114715847 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05f413fd40 0x7f05e00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 10 monclient(hunting): picked mon.c con 0x7f05f413fd40 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 1 --2- 172.21.15.132:0/114715847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05e0004110 0x7f05e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 10 monclient(hunting): picked mon.b con 0x7f05e0004110 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 1 --2- 172.21.15.132:0/114715847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05e0004f40 0x7f05e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 10 monclient(hunting): picked mon.a con 0x7f05e0004f40 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 1 -- 172.21.15.132:0/114715847 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f40b13c0 con 0x7f05e0004f40 2024-03-13T07:34:54.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 1 -- 172.21.15.132:0/114715847 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f40b16a0 con 0x7f05e0004110 2024-03-13T07:34:54.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 1 -- 172.21.15.132:0/114715847 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e000bf00 con 0x7f05f413fd40 2024-03-13T07:34:54.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f05f27fc640 10 monclient(hunting): _finish_auth 0 2024-03-13T07:34:54.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f88fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f88fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.642+0000 7f88fffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-03-13T07:34:54.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.644+0000 7f3e2effd640 1 -- 172.21.15.132:0/1796523488 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e30017070 con 0x7f3e4006f950 2024-03-13T07:34:54.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.644+0000 7f3e2effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.644+0000 7f3e2effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:54.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.644+0000 7f3e2effd640 10 monclient: dump: 2024-03-13T07:34:54.651 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.651 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.651 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.651 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.651 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.651 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.651 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.651 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.651 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.651 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.644+0000 7f3e2effd640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.644+0000 7f3e2effd640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.644+0000 7f3e2effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.644+0000 7f3e44ec1640 5 monclient: authenticate success, global_id 4308 2024-03-13T07:34:54.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.644+0000 7f3e44ec1640 10 monclient: _renew_subs 2024-03-13T07:34:54.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.644+0000 7f3e44ec1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.644+0000 7f3e44ec1640 1 -- 172.21.15.132:0/1796523488 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4014a1c0 con 0x7f3e4006f950 2024-03-13T07:34:54.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.644+0000 7f3e44ec1640 10 client.4308.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.644+0000 7f3e44ec1640 10 monclient: _renew_subs 2024-03-13T07:34:54.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.644+0000 7f3e44ec1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.644+0000 7f3e44ec1640 1 -- 172.21.15.132:0/1796523488 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e40149e50 con 0x7f3e4006f950 2024-03-13T07:34:54.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.644+0000 7f3e44ec1640 1 librados: init done 2024-03-13T07:34:54.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.645+0000 7f3e2effd640 1 -- 172.21.15.132:0/1796523488 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e300052d0 con 0x7f3e4006f950 2024-03-13T07:34:54.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.645+0000 7f3e2effd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.645+0000 7f3e2effd640 1 -- 172.21.15.132:0/1796523488 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e30003830 con 0x7f3e4006f950 2024-03-13T07:34:54.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.645+0000 7f3e2effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.645+0000 7f3e3cd99640 4 set_mon_vals no callback set 2024-03-13T07:34:54.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.645+0000 7f3e2effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:54.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.645+0000 7f3e2effd640 10 monclient: dump: 2024-03-13T07:34:54.653 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.653 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.653 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.653 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.653 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.653 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.653 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.653 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.653 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.653 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.646+0000 7f3e44ec1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.646+0000 7f3e44ec1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.646+0000 7f3e44ec1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.646+0000 7f3e44ec1640 1 -- 172.21.15.132:0/1796523488 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e24005080 con 0x7f3e4006f950 2024-03-13T07:34:54.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.652+0000 7f88fffff640 10 monclient(hunting): handle_auth_done global_id 4311 payload 293 2024-03-13T07:34:54.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.652+0000 7f88fffff640 1 -- 172.21.15.132:0/4056211493 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88e8004800 msgr2=0x7f88e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.652+0000 7f88fffff640 1 --2- 172.21.15.132:0/4056211493 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88e8004800 0x7f88e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.652+0000 7f88fffff640 1 -- 172.21.15.132:0/4056211493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88e8005410 msgr2=0x7f88e800ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.652+0000 7f88fffff640 1 --2- 172.21.15.132:0/4056211493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88e8005410 0x7f88e800ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.652+0000 7f88fffff640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.652+0000 7f88fffff640 1 monclient: found mon.a 2024-03-13T07:34:54.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.652+0000 7f88fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.652+0000 7f88fffff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.652+0000 7f88fffff640 1 -- 172.21.15.132:0/4056211493 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e8003520 con 0x7f88e8003a00 2024-03-13T07:34:54.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.652+0000 7f88fffff640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.652+0000 7f88fffff640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.652+0000 7f88fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.652+0000 7f8904ed2640 1 -- 172.21.15.132:0/4056211493 reap_dead start 2024-03-13T07:34:54.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.669+0000 7fb34699a640 10 monclient: get_monmap_and_config 2024-03-13T07:34:54.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.669+0000 7fb34699a640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.669+0000 7fb34699a640 10 monclient: monmap: 2024-03-13T07:34:54.670 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.670 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.670 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:54.670819+0000 2024-03-13T07:34:54.670 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:54.670819+0000 2024-03-13T07:34:54.670 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.670 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.670 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:54.670 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:54.670 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:54.670 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:54.670 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:54.670 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.669+0000 7fb34699a640 1 Processor -- start 2024-03-13T07:34:54.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.669+0000 7fb34699a640 1 -- start start 2024-03-13T07:34:54.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.669+0000 7fb34699a640 10 monclient: init 2024-03-13T07:34:54.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.670+0000 7fb34699a640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.670+0000 7fb34699a640 10 monclient: _add_conns ranks=[4,3,0,2,1] 2024-03-13T07:34:54.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.670+0000 7fb34699a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb3400b1e70 0x7fb3400b2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.670+0000 7fb34699a640 10 monclient(hunting): picked mon.noname-c con 0x7fb3400b1e70 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:54.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.670+0000 7fb34699a640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb3400b00f0 0x7fb34013fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.670+0000 7fb34699a640 10 monclient(hunting): picked mon.noname-a con 0x7fb3400b00f0 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:54.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.670+0000 7fb34699a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb340140400 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:54.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.670+0000 7fb34699a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3400b1390 con 0x7fb340140400 2024-03-13T07:34:54.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.670+0000 7fb34699a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.670+0000 7fb34699a640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3400b1670 con 0x7fb3400b00f0 2024-03-13T07:34:54.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.670+0000 7fb34699a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.670+0000 7fb34699a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3400b05a0 con 0x7fb3400b1e70 2024-03-13T07:34:54.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.670+0000 7fb34699a640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.670+0000 7fb34699a640 10 monclient(hunting): authenticate will time out at 1322.079537s 2024-03-13T07:34:54.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.687+0000 7f47257fa640 1 -- 172.21.15.132:0/1826027506 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f47140567c0 con 0x7f47280aea20 2024-03-13T07:34:54.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.687+0000 7f47257fa640 10 client.4307.objecter ms_dispatch 0x7f4728000b90 osd_map(25..25 src has 1..25) v4 2024-03-13T07:34:54.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.687+0000 7f47257fa640 3 client.4307.objecter handle_osd_map got epochs [25,25] > 0 2024-03-13T07:34:54.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.687+0000 7f47257fa640 3 client.4307.objecter handle_osd_map decoding full epoch 25 2024-03-13T07:34:54.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.687+0000 7f47257fa640 20 client.4307.objecter dump_active .. 0 homeless 2024-03-13T07:34:54.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.845+0000 7fcefeffd640 1 -- 172.21.15.132:0/883543987 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fcf00033cf0 con 0x7fcf1013bbb0 2024-03-13T07:34:54.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.845+0000 7fcefeffd640 1 --2- 172.21.15.132:0/883543987 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcef006d5b0 0x7fcef006fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.845+0000 7fcf0e59c640 1 --2- 172.21.15.132:0/883543987 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcef006d5b0 0x7fcef006fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.845+0000 7fcf0e59c640 10 monclient: get_auth_request con 0x7fcef006d5b0 auth_method 0 2024-03-13T07:34:54.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.846+0000 7fcf0e59c640 1 --2- 172.21.15.132:0/883543987 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcef006d5b0 0x7fcef006fa70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009710 tx=0x7fcf04006c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.846+0000 7fcefeffd640 10 client.4309.objecter ms_handle_connect 0x7fcef006d5b0 2024-03-13T07:34:54.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.850+0000 7fcefeffd640 1 -- 172.21.15.132:0/883543987 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcf00079c00 con 0x7fcf1013bbb0 2024-03-13T07:34:54.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.850+0000 7fcefeffd640 10 client.4309.objecter ms_dispatch 0x7fcf10000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:54.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.850+0000 7fcefeffd640 3 client.4309.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:54.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.850+0000 7fcefeffd640 3 client.4309.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:54.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.850+0000 7fcefeffd640 20 client.4309.objecter dump_active .. 0 homeless 2024-03-13T07:34:54.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.861+0000 7f2d4a59c640 1 --2- 172.21.15.132:0/4253934313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d4c0b5320 0x7f2d4c0b0220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0b90f0 tx=0x7f2d3c02b7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1bde1f46b86e638 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.861+0000 7fcefeffd640 1 -- 172.21.15.132:0/883543987 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fcf00035e20 con 0x7fcf1013bbb0 2024-03-13T07:34:54.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.861+0000 7fcefeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.861+0000 7fcefeffd640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:54.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.873+0000 7fb33ffff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb3400b1e70 0x7fb3400b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.873+0000 7f2d49d9b640 1 -- 172.21.15.132:0/4253934313 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c038070 con 0x7f2d4c0b5320 2024-03-13T07:34:54.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.873+0000 7f2d49d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d49d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:54.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d49d9b640 10 monclient: dump: 2024-03-13T07:34:54.875 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.875 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.875 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.875 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.875 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.875 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.875 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.875 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.875 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.876 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d49d9b640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d49d9b640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d49d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d49d9b640 1 -- 172.21.15.132:0/4253934313 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3c02f050 con 0x7f2d4c0b5320 2024-03-13T07:34:54.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d50ec2640 5 monclient: authenticate success, global_id 4315 2024-03-13T07:34:54.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d50ec2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:54.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d49d9b640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d50ec2640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:54.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d49d9b640 1 -- 172.21.15.132:0/4253934313 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c031970 con 0x7f2d4c0b5320 2024-03-13T07:34:54.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d50ec2640 4 set_mon_vals no callback set 2024-03-13T07:34:54.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d49d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d49d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:54.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d49d9b640 10 monclient: dump: 2024-03-13T07:34:54.877 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.877 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.877 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.877 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.877 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.877 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.877 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.877 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.877 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.877 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7fb33ffff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb3400b1e70 0x7fb3400b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:48998/0 (socket says 172.21.15.132:48998) 2024-03-13T07:34:54.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7fb33ffff640 1 -- 172.21.15.132:0/859552283 learned_addr learned my addr 172.21.15.132:0/859552283 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7fb33ffff640 10 monclient(hunting): get_auth_request con 0x7fb3400b1e70 auth_method 0 2024-03-13T07:34:54.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7fb33ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7fb33ffff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7fb33ffff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d50ec2640 10 monclient: shutdown 2024-03-13T07:34:54.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d50ec2640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d50ec2640 1 -- 172.21.15.132:0/4253934313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d4c0b5320 msgr2=0x7f2d4c0b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.874+0000 7f2d50ec2640 1 --2- 172.21.15.132:0/4253934313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d4c0b5320 0x7f2d4c0b0220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0b90f0 tx=0x7f2d3c02b7f0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.875+0000 7f2d50ec2640 1 -- 172.21.15.132:0/4253934313 shutdown_connections 2024-03-13T07:34:54.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.875+0000 7f2d50ec2640 1 --2- 172.21.15.132:0/4253934313 >> v2:172.21.15.132:3300/0 conn(0x7f2d4c13fd40 0x7f2d4c0b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.875+0000 7f2d50ec2640 1 --2- 172.21.15.132:0/4253934313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d4c0b5320 0x7f2d4c0b0220 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.875+0000 7f2d50ec2640 1 -- 172.21.15.132:0/4253934313 >> 172.21.15.132:0/4253934313 conn(0x7f2d4c06d2b0 msgr2=0x7f2d4c06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.875+0000 7f2d50ec2640 1 -- 172.21.15.132:0/4253934313 shutdown_connections 2024-03-13T07:34:54.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.875+0000 7f2d50ec2640 1 -- 172.21.15.132:0/4253934313 wait complete. 2024-03-13T07:34:54.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.876+0000 7f05ebfff640 1 --2- 172.21.15.132:0/114715847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05e0004110 0x7f05e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.876+0000 7f2d50ec2640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.876+0000 7f2d50ec2640 10 monclient: monmap: 2024-03-13T07:34:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.881 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:54.882 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.876+0000 7fb33ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.876+0000 7fb33ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.876+0000 7fb33ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.876+0000 7f2d50ec2640 1 librados: starting msgr at 2024-03-13T07:34:54.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.876+0000 7f2d50ec2640 1 librados: starting objecter 2024-03-13T07:34:54.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.876+0000 7f2d50ec2640 1 Processor -- start 2024-03-13T07:34:54.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.876+0000 7f2d50ec2640 1 -- start start 2024-03-13T07:34:54.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.876+0000 7f2d50ec2640 1 librados: setting wanted keys 2024-03-13T07:34:54.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.876+0000 7f2d50ec2640 1 librados: calling monclient init 2024-03-13T07:34:54.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.876+0000 7f2d50ec2640 10 monclient: init 2024-03-13T07:34:54.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f05ebfff640 10 monclient(hunting): get_auth_request con 0x7f05e0004110 auth_method 0 2024-03-13T07:34:54.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f05ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f05ebfff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f05ebfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-03-13T07:34:54.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f2d50ec2640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f2d50ec2640 10 monclient: _add_conns ranks=[2,1,0] 2024-03-13T07:34:54.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f2d50ec2640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d4c0b1ac0 0x7f2d4c138530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f2d50ec2640 10 monclient(hunting): picked mon.noname-c con 0x7f2d4c0b1ac0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f2d50ec2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d4c0b5320 0x7f2d4c1403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f2d50ec2640 10 monclient(hunting): picked mon.noname-b con 0x7f2d4c0b5320 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f2d50ec2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d4c13fd40 0x7f2d4c141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f2d50ec2640 10 monclient(hunting): picked mon.noname-a con 0x7f2d4c13fd40 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f2d50ec2640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f2d50ec2640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0b4980 con 0x7f2d4c13fd40 2024-03-13T07:34:54.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f2d50ec2640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f2d50ec2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0b4800 con 0x7f2d4c0b5320 2024-03-13T07:34:54.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f2d50ec2640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f2d50ec2640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c14cef0 con 0x7f2d4c0b1ac0 2024-03-13T07:34:54.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f2d50ec2640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.877+0000 7f2d50ec2640 10 monclient(hunting): authenticate will time out at 1322.286926s 2024-03-13T07:34:54.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.879+0000 7fb33ffff640 10 monclient(hunting): handle_auth_done global_id 4318 payload 274 2024-03-13T07:34:54.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.879+0000 7fb33ffff640 1 -- 172.21.15.132:0/859552283 >> v2:172.21.15.132:3300/0 conn(0x7fb3400b00f0 msgr2=0x7fb34013fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.879+0000 7fb33ffff640 1 --2- 172.21.15.132:0/859552283 >> v2:172.21.15.132:3300/0 conn(0x7fb3400b00f0 0x7fb34013fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.879+0000 7fb33ffff640 1 -- 172.21.15.132:0/859552283 >> v1:172.21.15.132:6789/0 conn(0x7fb340140400 legacy=0x7fb340142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.879+0000 7fb33ffff640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.879+0000 7fb33ffff640 1 monclient: found mon.noname-c 2024-03-13T07:34:54.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.880+0000 7fb33ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.880+0000 7fb33ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.880+0000 7fb33ffff640 1 -- 172.21.15.132:0/859552283 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3400b2790 con 0x7fb3400b1e70 2024-03-13T07:34:54.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.880+0000 7f05ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.880+0000 7f05ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.880+0000 7f05ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-03-13T07:34:54.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.881+0000 7f47257fa640 1 -- 172.21.15.132:0/1826027506 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f4714021020 con 0x7f47280aea20 2024-03-13T07:34:54.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.881+0000 7f47257fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.881+0000 7f47257fa640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:54.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.882+0000 7f05f2ffd640 1 --2- 172.21.15.132:0/114715847 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05f413fd40 0x7f05e00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.885+0000 7f05ebfff640 10 monclient(hunting): handle_auth_done global_id 4314 payload 293 2024-03-13T07:34:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.885+0000 7f05ebfff640 1 -- 172.21.15.132:0/114715847 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05f413fd40 msgr2=0x7f05e00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.885+0000 7f05ebfff640 1 --2- 172.21.15.132:0/114715847 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05f413fd40 0x7f05e00039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.885+0000 7f05ebfff640 1 -- 172.21.15.132:0/114715847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05e0004f40 msgr2=0x7f05e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.885+0000 7f05ebfff640 1 --2- 172.21.15.132:0/114715847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05e0004f40 0x7f05e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.885+0000 7f05ebfff640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.885+0000 7f05ebfff640 1 monclient: found mon.b 2024-03-13T07:34:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.885+0000 7f05ebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.885+0000 7f05ebfff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.885+0000 7f05ebfff640 1 -- 172.21.15.132:0/114715847 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e0003520 con 0x7f05e0004110 2024-03-13T07:34:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.885+0000 7f05ebfff640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.885+0000 7f05ebfff640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.885+0000 7f05ebfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.886+0000 7f05f9974640 5 monclient: authenticate success, global_id 4314 2024-03-13T07:34:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.886+0000 7f05f9974640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:54.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.888+0000 7f05ebfff640 1 --2- 172.21.15.132:0/114715847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05e0004110 0x7f05e0004800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f05dc001050 tx=0x7f05e0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f87bff48477db57a server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.894+0000 7fb33ffff640 1 --2- 172.21.15.132:0/859552283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3400b1e70 0x7fb3400b2250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb34006ba90 tx=0x7fb33002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19e989437ebf943e server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.895+0000 7f2d43fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d4c0b5320 0x7f2d4c1403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.895+0000 7f2d43fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d4c0b5320 0x7f2d4c1403e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:49000/0 (socket says 172.21.15.132:49000) 2024-03-13T07:34:54.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.895+0000 7f2d43fff640 1 -- 172.21.15.132:0/4088929904 learned_addr learned my addr 172.21.15.132:0/4088929904 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.895+0000 7f2d43fff640 10 monclient(hunting): get_auth_request con 0x7f2d4c0b5320 auth_method 0 2024-03-13T07:34:54.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.895+0000 7f2d43fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.895+0000 7f2d43fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.895+0000 7f2d43fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f27fc640 1 -- 172.21.15.132:0/114715847 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc007ca0 con 0x7f05e0004110 2024-03-13T07:34:54.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:54.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f27fc640 10 monclient: dump: 2024-03-13T07:34:54.900 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.900 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.900 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.900 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.900 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.900 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.900 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.900 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.900 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.901 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f27fc640 1 -- 172.21.15.132:0/114715847 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc030070 con 0x7f05e0004110 2024-03-13T07:34:54.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f9974640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:54.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f27fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f27fc640 1 -- 172.21.15.132:0/114715847 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc005d10 con 0x7f05e0004110 2024-03-13T07:34:54.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f9974640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:54.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f9974640 4 set_mon_vals no callback set 2024-03-13T07:34:54.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:54.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f27fc640 10 monclient: dump: 2024-03-13T07:34:54.901 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.902 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.902 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.902 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.902 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.902 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.902 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.902 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.902 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.902 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f2d43fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f2d43fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f2d43fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f9974640 10 monclient: shutdown 2024-03-13T07:34:54.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f9974640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f9974640 1 -- 172.21.15.132:0/114715847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05e0004110 msgr2=0x7f05e0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f9974640 1 --2- 172.21.15.132:0/114715847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05e0004110 0x7f05e0004800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f05dc001050 tx=0x7f05e0005830 comp rx=0 tx=0).stop 2024-03-13T07:34:54.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f37fe640 1 -- 172.21.15.132:0/114715847 reap_dead start 2024-03-13T07:34:54.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f9974640 1 -- 172.21.15.132:0/114715847 shutdown_connections 2024-03-13T07:34:54.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.899+0000 7f05f9974640 1 -- 172.21.15.132:0/114715847 >> 172.21.15.132:0/114715847 conn(0x7f05f406d2b0 msgr2=0x7f05f406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.900+0000 7fb33effd640 1 -- 172.21.15.132:0/859552283 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330035020 con 0x7fb3400b1e70 2024-03-13T07:34:54.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.900+0000 7fb33effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.900+0000 7fb33effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:54.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.900+0000 7fb33effd640 10 monclient: dump: 2024-03-13T07:34:54.905 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.905 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.905 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.905 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.905 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.906 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.906 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.906 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.906 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.906 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.900+0000 7fb33effd640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.900+0000 7fb33effd640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.900+0000 7fb33effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.900+0000 7fb33effd640 1 -- 172.21.15.132:0/859552283 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb330031870 con 0x7fb3400b1e70 2024-03-13T07:34:54.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.900+0000 7fb34699a640 5 monclient: authenticate success, global_id 4318 2024-03-13T07:34:54.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.900+0000 7fb34699a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:54.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.900+0000 7f05f9974640 1 -- 172.21.15.132:0/114715847 shutdown_connections 2024-03-13T07:34:54.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.900+0000 7f05f9974640 1 -- 172.21.15.132:0/114715847 wait complete. 2024-03-13T07:34:54.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.901+0000 7f05f9974640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.901+0000 7f05f9974640 10 monclient: monmap: 2024-03-13T07:34:54.907 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.908 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:54.908 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:54.908 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:54.908 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.908 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.908 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:54.908 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:54.908 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:54.908 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.901+0000 7f05f9974640 1 librados: starting msgr at 2024-03-13T07:34:54.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.901+0000 7f05f9974640 1 librados: starting objecter 2024-03-13T07:34:54.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.901+0000 7f05f9974640 1 Processor -- start 2024-03-13T07:34:54.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.901+0000 7fb33effd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.901+0000 7fb33effd640 1 -- 172.21.15.132:0/859552283 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330031c40 con 0x7fb3400b1e70 2024-03-13T07:34:54.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.901+0000 7fb33effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.901+0000 7fb33effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:54.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.901+0000 7fb33effd640 10 monclient: dump: 2024-03-13T07:34:54.910 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.910 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.910 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.910 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.910 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.910 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.910 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.910 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.910 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.910 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.901+0000 7fb34699a640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:54.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.901+0000 7fb34699a640 4 set_mon_vals no callback set 2024-03-13T07:34:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.902+0000 7fb34699a640 10 monclient: shutdown 2024-03-13T07:34:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.902+0000 7fb34699a640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.902+0000 7fb34699a640 1 -- 172.21.15.132:0/859552283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3400b1e70 msgr2=0x7fb3400b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.902+0000 7fb34699a640 1 --2- 172.21.15.132:0/859552283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3400b1e70 0x7fb3400b2250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb34006ba90 tx=0x7fb33002b910 comp rx=0 tx=0).stop 2024-03-13T07:34:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.902+0000 7f05f9974640 1 -- start start 2024-03-13T07:34:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.902+0000 7f05f9974640 1 librados: setting wanted keys 2024-03-13T07:34:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.902+0000 7f05f9974640 1 librados: calling monclient init 2024-03-13T07:34:54.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.902+0000 7f05f9974640 10 monclient: init 2024-03-13T07:34:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.902+0000 7fb34699a640 1 -- 172.21.15.132:0/859552283 shutdown_connections 2024-03-13T07:34:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.902+0000 7fb34699a640 1 --2- 172.21.15.132:0/859552283 >> v2:172.21.15.132:3300/0 conn(0x7fb3400b00f0 0x7fb34013fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.902+0000 7fb34699a640 1 --2- 172.21.15.132:0/859552283 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3400b1e70 0x7fb3400b2250 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.902+0000 7fb34699a640 1 -- 172.21.15.132:0/859552283 >> 172.21.15.132:0/859552283 conn(0x7fb3400b80e0 msgr2=0x7fb3400b84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f2d43fff640 10 monclient(hunting): handle_auth_done global_id 4321 payload 824 2024-03-13T07:34:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f2d43fff640 1 -- 172.21.15.132:0/4088929904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d4c0b1ac0 msgr2=0x7f2d4c138530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f2d43fff640 1 --2- 172.21.15.132:0/4088929904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d4c0b1ac0 0x7f2d4c138530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f2d43fff640 1 -- 172.21.15.132:0/4088929904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d4c13fd40 msgr2=0x7f2d4c141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f2d43fff640 1 --2- 172.21.15.132:0/4088929904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d4c13fd40 0x7f2d4c141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f2d43fff640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f2d43fff640 1 monclient: found mon.noname-b 2024-03-13T07:34:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f2d43fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f2d43fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f2d43fff640 1 -- 172.21.15.132:0/4088929904 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c0aa0f0 con 0x7f2d4c0b5320 2024-03-13T07:34:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f05f9974640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f05f9974640 10 monclient: _add_conns ranks=[1,0,2] 2024-03-13T07:34:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f05f9974640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05e0004110 0x7f05f41403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f05f9974640 10 monclient(hunting): picked mon.noname-b con 0x7f05e0004110 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f05f9974640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05f40a94d0 0x7f05f4138300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7fb34699a640 1 -- 172.21.15.132:0/859552283 shutdown_connections 2024-03-13T07:34:54.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7fb34699a640 1 -- 172.21.15.132:0/859552283 wait complete. 2024-03-13T07:34:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f05f9974640 10 monclient(hunting): picked mon.noname-a con 0x7f05f40a94d0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f05f9974640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05f4138840 0x7f05f41d82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f05f9974640 10 monclient(hunting): picked mon.noname-c con 0x7f05f4138840 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f05f9974640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f05f9974640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f40b4980 con 0x7f05f40a94d0 2024-03-13T07:34:54.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f05f9974640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.904+0000 7fb34699a640 10 monclient: build_initial_monmap 2024-03-13T07:34:54.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f05f9974640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f40b4800 con 0x7f05e0004110 2024-03-13T07:34:54.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f05f9974640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.904+0000 7fb34699a640 10 monclient: monmap: 2024-03-13T07:34:54.918 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:54.918 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:54.918 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:54.918 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:54.918 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:54.918 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.918 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:54.918 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:54.919 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:54.919 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f05f9974640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f40b1f30 con 0x7f05f4138840 2024-03-13T07:34:54.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f05f9974640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.903+0000 7f05f9974640 10 monclient(hunting): authenticate will time out at 1322.312878s 2024-03-13T07:34:54.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.905+0000 7fb34699a640 1 librados: starting msgr at 2024-03-13T07:34:54.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.905+0000 7fb34699a640 1 librados: starting objecter 2024-03-13T07:34:54.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.905+0000 7fb34699a640 1 Processor -- start 2024-03-13T07:34:54.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.905+0000 7fb34699a640 1 -- start start 2024-03-13T07:34:54.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.905+0000 7fb34699a640 1 librados: setting wanted keys 2024-03-13T07:34:54.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.905+0000 7fb34699a640 1 librados: calling monclient init 2024-03-13T07:34:54.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.905+0000 7fb34699a640 10 monclient: init 2024-03-13T07:34:54.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.906+0000 7fb34699a640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:54.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.906+0000 7fb34699a640 10 monclient: _add_conns ranks=[2,0,1] 2024-03-13T07:34:54.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.906+0000 7fb34699a640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3400b00f0 0x7fb34013e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.906+0000 7fb34699a640 10 monclient(hunting): picked mon.noname-c con 0x7fb3400b00f0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:54.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.906+0000 7fb34699a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3400b1e70 0x7fb3400aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.906+0000 7fb34699a640 10 monclient(hunting): picked mon.noname-a con 0x7fb3400b1e70 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:54.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.906+0000 7fb34699a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb340140400 0x7fb3400ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.906+0000 7fb34699a640 10 monclient(hunting): picked mon.noname-b con 0x7fb340140400 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:54.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.906+0000 7fb34699a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.907+0000 7fb34699a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3401435f0 con 0x7fb3400b1e70 2024-03-13T07:34:54.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.907+0000 7fb34699a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.907+0000 7fb34699a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb340143470 con 0x7fb340140400 2024-03-13T07:34:54.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.907+0000 7fb34699a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:54.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.907+0000 7fb34699a640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb340143770 con 0x7fb3400b00f0 2024-03-13T07:34:54.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.907+0000 7fb34699a640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:54.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.907+0000 7fb34699a640 10 monclient(hunting): authenticate will time out at 1322.316226s 2024-03-13T07:34:54.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.907+0000 7f2d43fff640 1 --2- 172.21.15.132:0/4088929904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d4c0b5320 0x7f2d4c1403e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400c410 tx=0x7f2d3400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.907+0000 7f2d42ffd640 10 client.?.objecter ms_handle_connect 0x7f2d4c0b5320 2024-03-13T07:34:54.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.907+0000 7f2d42ffd640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:54.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.910+0000 7f05f2ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05e0004110 0x7f05f41403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.910+0000 7f05f2ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05e0004110 0x7f05f41403e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:49014/0 (socket says 172.21.15.132:49014) 2024-03-13T07:34:54.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.910+0000 7f05f2ffd640 1 -- 172.21.15.132:0/1573811828 learned_addr learned my addr 172.21.15.132:0/1573811828 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.910+0000 7f05f2ffd640 10 monclient(hunting): get_auth_request con 0x7f05e0004110 auth_method 0 2024-03-13T07:34:54.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.910+0000 7f05f2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.910+0000 7f05f2ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.910+0000 7f05f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.927 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.911+0000 7fb344f37640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb340140400 0x7fb3400ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.927 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.911+0000 7fb344f37640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb340140400 0x7fb3400ab5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:49026/0 (socket says 172.21.15.132:49026) 2024-03-13T07:34:54.927 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.911+0000 7fb344f37640 1 -- 172.21.15.132:0/2065159372 learned_addr learned my addr 172.21.15.132:0/2065159372 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:54.927 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.911+0000 7fb344f37640 10 monclient(hunting): get_auth_request con 0x7fb340140400 auth_method 0 2024-03-13T07:34:54.927 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.911+0000 7fb344f37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:54.927 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.911+0000 7fb344f37640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:54.927 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.911+0000 7fb344f37640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:54.928 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f05f2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.928 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f05f2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.928 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f05f2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d42ffd640 1 -- 172.21.15.132:0/4088929904 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d34017070 con 0x7f2d4c0b5320 2024-03-13T07:34:54.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d42ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d42ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:54.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d42ffd640 10 monclient: dump: 2024-03-13T07:34:54.929 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.929 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.929 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.929 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.929 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.929 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.929 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.929 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.929 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.930 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d42ffd640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d42ffd640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d42ffd640 1 -- 172.21.15.132:0/4088929904 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d340035f0 con 0x7f2d4c0b5320 2024-03-13T07:34:54.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d42ffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d42ffd640 1 -- 172.21.15.132:0/4088929904 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d34003a20 con 0x7f2d4c0b5320 2024-03-13T07:34:54.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d42ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d42ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:54.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d42ffd640 10 monclient: dump: 2024-03-13T07:34:54.930 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.930 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.930 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.931 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.931 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.931 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.931 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.931 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.931 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.931 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d4959a640 4 set_mon_vals no callback set 2024-03-13T07:34:54.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d50ec2640 5 monclient: authenticate success, global_id 4321 2024-03-13T07:34:54.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d50ec2640 10 monclient: _renew_subs 2024-03-13T07:34:54.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d50ec2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d50ec2640 1 -- 172.21.15.132:0/4088929904 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4c0aa3d0 con 0x7f2d4c0b5320 2024-03-13T07:34:54.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d50ec2640 10 client.4321.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d50ec2640 10 monclient: _renew_subs 2024-03-13T07:34:54.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d50ec2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d50ec2640 1 -- 172.21.15.132:0/4088929904 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4c142c40 con 0x7f2d4c0b5320 2024-03-13T07:34:54.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.912+0000 7f2d50ec2640 1 librados: init done 2024-03-13T07:34:54.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.913+0000 7fb344f37640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:54.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.913+0000 7fb344f37640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:54.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.913+0000 7fb344f37640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:54.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.913+0000 7f2d50ec2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.913+0000 7f2d50ec2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.913+0000 7f2d50ec2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.913+0000 7f2d50ec2640 1 -- 172.21.15.132:0/4088929904 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d4c04f000 con 0x7f2d4c0b5320 2024-03-13T07:34:54.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.915+0000 7f05f2ffd640 10 monclient(hunting): handle_auth_done global_id 4324 payload 824 2024-03-13T07:34:54.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.915+0000 7f05f2ffd640 1 -- 172.21.15.132:0/1573811828 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05f4138840 msgr2=0x7f05f41d82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.915+0000 7f05f2ffd640 1 --2- 172.21.15.132:0/1573811828 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05f4138840 0x7f05f41d82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.915+0000 7f05f2ffd640 1 -- 172.21.15.132:0/1573811828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05f40a94d0 msgr2=0x7f05f4138300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.915+0000 7f05f2ffd640 1 --2- 172.21.15.132:0/1573811828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05f40a94d0 0x7f05f4138300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.915+0000 7f05f2ffd640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.915+0000 7f05f2ffd640 1 monclient: found mon.noname-b 2024-03-13T07:34:54.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.915+0000 7f05f2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.915+0000 7f05f2ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.915+0000 7f05f2ffd640 1 -- 172.21.15.132:0/1573811828 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f4143330 con 0x7f05e0004110 2024-03-13T07:34:54.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42aaffd640 1 -- 172.21.15.132:0/547973600 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac019070 con 0x7f42b40b5320 2024-03-13T07:34:54.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42aaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:54.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42aaffd640 10 monclient: dump: 2024-03-13T07:34:54.934 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.934 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.934 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.934 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.934 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.935 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.935 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.935 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.935 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.935 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42aaffd640 10 monclient: _finish_auth 0 2024-03-13T07:34:54.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42aaffd640 10 monclient: _check_auth_tickets 2024-03-13T07:34:54.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:54.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42aaffd640 1 -- 172.21.15.132:0/547973600 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac00cba0 con 0x7f42b40b5320 2024-03-13T07:34:54.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42aaffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:54.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42aaffd640 1 -- 172.21.15.132:0/547973600 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac014440 con 0x7f42b40b5320 2024-03-13T07:34:54.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:54.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42aaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:54.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42aaffd640 10 monclient: dump: 2024-03-13T07:34:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:54.936 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42b8aba640 4 set_mon_vals no callback set 2024-03-13T07:34:54.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f88fffff640 1 --2- 172.21.15.132:0/4056211493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88e8003a00 0x7f88e8003e00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f88f000c190 tx=0x7f88e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d61834a64618003 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42bc521640 5 monclient: authenticate success, global_id 4317 2024-03-13T07:34:54.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42bc521640 10 monclient: _renew_subs 2024-03-13T07:34:54.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42bc521640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.916+0000 7f42bc521640 1 -- 172.21.15.132:0/547973600 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b40ae630 con 0x7f42b40b5320 2024-03-13T07:34:54.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.917+0000 7f42bc521640 10 client.4317.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:54.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.917+0000 7f42bc521640 10 monclient: _renew_subs 2024-03-13T07:34:54.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.917+0000 7f42bc521640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.917+0000 7f42bc521640 1 -- 172.21.15.132:0/547973600 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b41cc310 con 0x7f42b40b5320 2024-03-13T07:34:54.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.917+0000 7f42bc521640 1 librados: init done 2024-03-13T07:34:54.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.918+0000 7f42bc521640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.918+0000 7f42bc521640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.918+0000 7f42bc521640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:54.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.918+0000 7f05f2ffd640 1 --2- 172.21.15.132:0/1573811828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05e0004110 0x7f05f41403e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f05e400c7f0 tx=0x7f05e400ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.918+0000 7f42bc521640 1 -- 172.21.15.132:0/547973600 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b413d1c0 con 0x7f42b40b5320 2024-03-13T07:34:54.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.918+0000 7f05f0ff9640 10 client.?.objecter ms_handle_connect 0x7f05e0004110 2024-03-13T07:34:54.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.918+0000 7f05f0ff9640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:54.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.926+0000 7fb344f37640 10 monclient(hunting): handle_auth_done global_id 4327 payload 824 2024-03-13T07:34:54.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.926+0000 7fb344f37640 1 -- 172.21.15.132:0/2065159372 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3400b00f0 msgr2=0x7fb34013e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.926+0000 7fb344f37640 1 --2- 172.21.15.132:0/2065159372 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3400b00f0 0x7fb34013e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.926+0000 7fb344f37640 1 -- 172.21.15.132:0/2065159372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3400b1e70 msgr2=0x7fb3400aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:54.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.926+0000 7fb344f37640 1 --2- 172.21.15.132:0/2065159372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3400b1e70 0x7fb3400aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:54.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.926+0000 7fb344f37640 10 monclient: _finish_hunting 0 2024-03-13T07:34:54.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.926+0000 7fb344f37640 1 monclient: found mon.noname-b 2024-03-13T07:34:54.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.926+0000 7fb344f37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:54.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.927+0000 7fb344f37640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:54.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.927+0000 7fb344f37640 1 -- 172.21.15.132:0/2065159372 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3400aadf0 con 0x7fb340140400 2024-03-13T07:34:54.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.930+0000 7f3e2effd640 1 -- 172.21.15.132:0/1796523488 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3e3001a070 con 0x7f3e4006f950 2024-03-13T07:34:54.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.930+0000 7f3e2effd640 1 --2- 172.21.15.132:0/1796523488 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3e2006d560 0x7f3e2006fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:54.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.931+0000 7f3e3e59c640 1 --2- 172.21.15.132:0/1796523488 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3e2006d560 0x7f3e2006fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:54.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.931+0000 7f3e3e59c640 10 monclient: get_auth_request con 0x7f3e2006d560 auth_method 0 2024-03-13T07:34:54.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.931+0000 7f3e3e59c640 1 --2- 172.21.15.132:0/1796523488 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3e2006d560 0x7f3e2006fa20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e380020a0 tx=0x7f3e38001690 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:54.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.931+0000 7f3e2effd640 10 client.4308.objecter ms_handle_connect 0x7f3e2006d560 2024-03-13T07:34:54.945 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.944+0000 7f3e2effd640 1 -- 172.21.15.132:0/1796523488 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3e300566f0 con 0x7f3e4006f950 2024-03-13T07:34:54.945 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.944+0000 7f3e2effd640 10 client.4308.objecter ms_dispatch 0x7f3e40000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:54.945 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.944+0000 7f3e2effd640 3 client.4308.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:54.945 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.944+0000 7f3e2effd640 3 client.4308.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:54.945 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.944+0000 7f3e2effd640 20 client.4308.objecter dump_active .. 0 homeless 2024-03-13T07:34:54.945 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.944+0000 7f3e2effd640 1 -- 172.21.15.132:0/1796523488 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f3e30021020 con 0x7f3e4006f950 2024-03-13T07:34:54.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.944+0000 7f3e2effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:54.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:54.944+0000 7f3e2effd640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:55.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.059+0000 7fcf14e4b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-03-13T07:34:55.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.060+0000 7fcf14e4b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-03-13T07:34:55.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.060+0000 7fcf14e4b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:55.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.060+0000 7fcf14e4b640 1 -- 172.21.15.132:0/883543987 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fcef4005640 con 0x7fcf1013bbb0 2024-03-13T07:34:55.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.110+0000 7f472edd8640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-03-13T07:34:55.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.110+0000 7f472edd8640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-03-13T07:34:55.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.110+0000 7f472edd8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:55.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.110+0000 7f472edd8640 1 -- 172.21.15.132:0/1826027506 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f472814b7a0 con 0x7f47280aea20 2024-03-13T07:34:55.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.119+0000 7f47257fa640 1 -- 172.21.15.132:0/1826027506 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4714058e20 con 0x7f47280aea20 2024-03-13T07:34:55.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.119+0000 7f47257fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-03-13T07:34:55.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.119+0000 7f47257fa640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:55.119 INFO:teuthology.orchestra.run.smithi132.stdout:77309411337 2024-03-13T07:34:55.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 10 librados: watch_flush enter 2024-03-13T07:34:55.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 10 librados: watch_flush exit 2024-03-13T07:34:55.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 20 client.4307.objecter shutdown clearing up homeless session... 2024-03-13T07:34:55.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 10 client.4307.objecter successfully canceled tick 2024-03-13T07:34:55.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 1 -- 172.21.15.132:0/1826027506 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f470c06d5b0 msgr2=0x7f470c06fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 1 --2- 172.21.15.132:0/1826027506 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f470c06d5b0 0x7f470c06fa70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f471c009990 tx=0x7f471c009110 comp rx=0 tx=0).stop 2024-03-13T07:34:55.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 10 monclient: shutdown 2024-03-13T07:34:55.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:55.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 1 -- 172.21.15.132:0/1826027506 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47280aea20 msgr2=0x7f472814baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 1 --2- 172.21.15.132:0/1826027506 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47280aea20 0x7f472814baf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f471400c440 tx=0x7f471400c840 comp rx=0 tx=0).stop 2024-03-13T07:34:55.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 1 -- 172.21.15.132:0/1826027506 shutdown_connections 2024-03-13T07:34:55.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 1 --2- 172.21.15.132:0/1826027506 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f470c06d5b0 0x7f470c06fa70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 1 --2- 172.21.15.132:0/1826027506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4728148c20 0x7f47280ab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 1 --2- 172.21.15.132:0/1826027506 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47280aea20 0x7f472814baf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 1 --2- 172.21.15.132:0/1826027506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f472806af30 0x7f4728071990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 1 -- 172.21.15.132:0/1826027506 >> 172.21.15.132:0/1826027506 conn(0x7f47280b4ed0 msgr2=0x7f47280b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:55.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 1 -- 172.21.15.132:0/1826027506 shutdown_connections 2024-03-13T07:34:55.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 1 -- 172.21.15.132:0/1826027506 wait complete. 2024-03-13T07:34:55.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.122+0000 7f472edd8640 1 librados: shutdown 2024-03-13T07:34:55.134 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411338 got 77309411337 for osd.4 2024-03-13T07:34:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.179+0000 7f3e44ec1640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2024-03-13T07:34:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.179+0000 7f3e44ec1640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2024-03-13T07:34:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.179+0000 7f3e44ec1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:55.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.179+0000 7f3e44ec1640 1 -- 172.21.15.132:0/1796523488 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3e24005640 con 0x7f3e4006f950 2024-03-13T07:34:55.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.181+0000 7f2d42ffd640 1 -- 172.21.15.132:0/4088929904 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2d3401a070 con 0x7f2d4c0b5320 2024-03-13T07:34:55.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.182+0000 7f2d42ffd640 1 --2- 172.21.15.132:0/4088929904 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2d2c06d5b0 0x7f2d2c06fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:55.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.182+0000 7f2d4a59c640 1 --2- 172.21.15.132:0/4088929904 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2d2c06d5b0 0x7f2d2c06fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:55.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.182+0000 7f2d4a59c640 10 monclient: get_auth_request con 0x7f2d2c06d5b0 auth_method 0 2024-03-13T07:34:55.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.182+0000 7f2d42ffd640 10 client.4321.objecter ms_handle_connect 0x7f2d2c06d5b0 2024-03-13T07:34:55.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.182+0000 7f2d4a59c640 1 --2- 172.21.15.132:0/4088929904 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2d2c06d5b0 0x7f2d2c06fa70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c035000 tx=0x7f2d3c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:55.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.185+0000 7f2d42ffd640 1 -- 172.21.15.132:0/4088929904 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2d34056810 con 0x7f2d4c0b5320 2024-03-13T07:34:55.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.185+0000 7f2d42ffd640 10 client.4321.objecter ms_dispatch 0x7f2d4c000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:55.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.185+0000 7f2d42ffd640 3 client.4321.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:55.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.185+0000 7f2d42ffd640 3 client.4321.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:55.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.185+0000 7f2d42ffd640 20 client.4321.objecter dump_active .. 0 homeless 2024-03-13T07:34:55.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.187+0000 7fb344f37640 1 --2- 172.21.15.132:0/2065159372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb340140400 0x7fb3400ab5e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb33800b990 tx=0x7fb33800be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:55.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.187+0000 7fb33cff9640 10 client.?.objecter ms_handle_connect 0x7fb340140400 2024-03-13T07:34:55.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.187+0000 7fb33cff9640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:55.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.197+0000 7f2d42ffd640 1 -- 172.21.15.132:0/4088929904 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f2d34021020 con 0x7f2d4c0b5320 2024-03-13T07:34:55.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.197+0000 7f2d42ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:55.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.197+0000 7f2d42ffd640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:55.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb33cff9640 1 -- 172.21.15.132:0/2065159372 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338019070 con 0x7fb340140400 2024-03-13T07:34:55.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb33cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:55.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb33cff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:55.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb33cff9640 10 monclient: dump: 2024-03-13T07:34:55.209 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:55.209 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:55.209 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:55.209 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:55.209 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:55.209 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:55.209 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:55.210 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:55.211 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:55.211 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:55.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb33cff9640 10 monclient: _finish_auth 0 2024-03-13T07:34:55.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb33cff9640 10 monclient: _check_auth_tickets 2024-03-13T07:34:55.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb33cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:55.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb33cff9640 1 -- 172.21.15.132:0/2065159372 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33800cba0 con 0x7fb340140400 2024-03-13T07:34:55.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb33cff9640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:55.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb33cff9640 1 -- 172.21.15.132:0/2065159372 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338014440 con 0x7fb340140400 2024-03-13T07:34:55.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb34699a640 5 monclient: authenticate success, global_id 4327 2024-03-13T07:34:55.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb34699a640 10 monclient: _renew_subs 2024-03-13T07:34:55.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb34699a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:55.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb34699a640 1 -- 172.21.15.132:0/2065159372 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3400acf10 con 0x7fb340140400 2024-03-13T07:34:55.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb33cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:55.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb33e7fc640 4 set_mon_vals no callback set 2024-03-13T07:34:55.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb33cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:55.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb33cff9640 10 monclient: dump: 2024-03-13T07:34:55.212 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:55.212 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:55.212 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:55.212 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:55.212 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:55.212 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:55.212 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:55.213 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:55.213 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:55.213 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:55.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb34699a640 10 client.4327.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:55.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb34699a640 10 monclient: _renew_subs 2024-03-13T07:34:55.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb34699a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:55.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb34699a640 1 -- 172.21.15.132:0/2065159372 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3400a94d0 con 0x7fb340140400 2024-03-13T07:34:55.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fb34699a640 1 librados: init done 2024-03-13T07:34:55.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fcefeffd640 1 -- 172.21.15.132:0/883543987 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcf0007c260 con 0x7fcf1013bbb0 2024-03-13T07:34:55.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fcefeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-03-13T07:34:55.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.208+0000 7fcefeffd640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:55.215 INFO:teuthology.orchestra.run.smithi132.stdout:55834574853 2024-03-13T07:34:55.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.209+0000 7fb34699a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:55.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.209+0000 7fb34699a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:55.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.209+0000 7fb34699a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:55.216 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.209+0000 7fb34699a640 1 -- 172.21.15.132:0/2065159372 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34004f000 con 0x7fb340140400 2024-03-13T07:34:55.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.210+0000 7fcf14e4b640 10 librados: watch_flush enter 2024-03-13T07:34:55.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.210+0000 7fcf14e4b640 10 librados: watch_flush exit 2024-03-13T07:34:55.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.210+0000 7fcf14e4b640 20 client.4309.objecter shutdown clearing up homeless session... 2024-03-13T07:34:55.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.210+0000 7fcf14e4b640 10 client.4309.objecter successfully canceled tick 2024-03-13T07:34:55.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.211+0000 7fcf14e4b640 1 -- 172.21.15.132:0/883543987 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcef006d5b0 msgr2=0x7fcef006fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.211+0000 7fcf14e4b640 1 --2- 172.21.15.132:0/883543987 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcef006d5b0 0x7fcef006fa70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009710 tx=0x7fcf04006c90 comp rx=0 tx=0).stop 2024-03-13T07:34:55.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.211+0000 7fcf14e4b640 10 monclient: shutdown 2024-03-13T07:34:55.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.211+0000 7fcf14e4b640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:55.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.211+0000 7fcf14e4b640 1 -- 172.21.15.132:0/883543987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf1013bbb0 msgr2=0x7fcf100b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.211+0000 7fcf14e4b640 1 --2- 172.21.15.132:0/883543987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf1013bbb0 0x7fcf100b2e20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcf00034000 tx=0x7fcf000046c0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.211+0000 7fcf14e4b640 1 -- 172.21.15.132:0/883543987 shutdown_connections 2024-03-13T07:34:55.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.211+0000 7fcf14e4b640 1 --2- 172.21.15.132:0/883543987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf1013bbb0 0x7fcf100b2e20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.211+0000 7fcf14e4b640 1 --2- 172.21.15.132:0/883543987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf10071d30 0x7fcf101db4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.211+0000 7fcf14e4b640 1 --2- 172.21.15.132:0/883543987 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcef006d5b0 0x7fcef006fa70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.211+0000 7fcf14e4b640 1 --2- 172.21.15.132:0/883543987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf1006ead0 0x7fcf10136d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.211+0000 7fcf14e4b640 1 -- 172.21.15.132:0/883543987 >> 172.21.15.132:0/883543987 conn(0x7fcf100b3e20 msgr2=0x7fcf10074530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:55.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.211+0000 7fcf14e4b640 1 -- 172.21.15.132:0/883543987 shutdown_connections 2024-03-13T07:34:55.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.211+0000 7fcf14e4b640 1 -- 172.21.15.132:0/883543987 wait complete. 2024-03-13T07:34:55.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.211+0000 7fcf14e4b640 1 librados: shutdown 2024-03-13T07:34:55.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.5 2024-03-13T07:34:55.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.259+0000 7f42aaffd640 1 -- 172.21.15.132:0/547973600 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f42ac01c070 con 0x7f42b40b5320 2024-03-13T07:34:55.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.259+0000 7f42aaffd640 1 --2- 172.21.15.132:0/547973600 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f429806d5b0 0x7f429806fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:55.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.259+0000 7f42ba2bd640 1 --2- 172.21.15.132:0/547973600 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f429806d5b0 0x7f429806fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:55.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.259+0000 7f42ba2bd640 10 monclient: get_auth_request con 0x7f429806d5b0 auth_method 0 2024-03-13T07:34:55.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.260+0000 7f42aaffd640 10 client.4317.objecter ms_handle_connect 0x7f429806d5b0 2024-03-13T07:34:55.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.260+0000 7f42ba2bd640 1 --2- 172.21.15.132:0/547973600 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f429806d5b0 0x7f429806fa70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f42a4001560 tx=0x7f42a4007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:55.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.274+0000 7f42aaffd640 1 -- 172.21.15.132:0/547973600 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f42ac057120 con 0x7f42b40b5320 2024-03-13T07:34:55.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.274+0000 7f42aaffd640 10 client.4317.objecter ms_dispatch 0x7f42b4000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:55.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.274+0000 7f42aaffd640 3 client.4317.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:55.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.274+0000 7f42aaffd640 3 client.4317.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:55.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.274+0000 7f42aaffd640 20 client.4317.objecter dump_active .. 0 homeless 2024-03-13T07:34:55.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.274+0000 7f42aaffd640 1 -- 172.21.15.132:0/547973600 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f42b413d1c0 con 0x7f42b40b5320 2024-03-13T07:34:55.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.274+0000 7f42aaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:55.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.274+0000 7f42aaffd640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:55.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.281+0000 7f88feffd640 1 -- 172.21.15.132:0/4056211493 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0007c60 con 0x7f88e8003a00 2024-03-13T07:34:55.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.281+0000 7f88feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:55.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.281+0000 7f88feffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:55.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.281+0000 7f88feffd640 10 monclient: dump: 2024-03-13T07:34:55.281 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:55.281 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:55.281 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:55.281 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:55.282 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:55.282 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:55.282 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:55.282 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:55.282 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:55.282 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:55.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.281+0000 7f88feffd640 1 -- 172.21.15.132:0/4056211493 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f0030070 con 0x7f88e8003a00 2024-03-13T07:34:55.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.281+0000 7f88feffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:55.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.281+0000 7f88feffd640 1 -- 172.21.15.132:0/4056211493 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0005d50 con 0x7f88e8003a00 2024-03-13T07:34:55.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.281+0000 7f8906935640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:55.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.281+0000 7f8906935640 4 set_mon_vals no callback set 2024-03-13T07:34:55.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.281+0000 7f88feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:55.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.281+0000 7f88feffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:55.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.281+0000 7f88feffd640 10 monclient: dump: 2024-03-13T07:34:55.282 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:55.283 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:55.283 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:55.283 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:55.283 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:55.283 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:55.283 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:55.283 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:55.284 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:55.284 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:55.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.282+0000 7f8906935640 10 monclient: shutdown 2024-03-13T07:34:55.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.282+0000 7f8906935640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:55.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.282+0000 7f8906935640 1 -- 172.21.15.132:0/4056211493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88e8003a00 msgr2=0x7f88e8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.282+0000 7f8906935640 1 --2- 172.21.15.132:0/4056211493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88e8003a00 0x7f88e8003e00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f88f000c190 tx=0x7f88e8005830 comp rx=0 tx=0).stop 2024-03-13T07:34:55.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.282+0000 7f8906935640 1 -- 172.21.15.132:0/4056211493 shutdown_connections 2024-03-13T07:34:55.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.282+0000 7f8906935640 1 --2- 172.21.15.132:0/4056211493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88e8003a00 0x7f88e8003e00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.282+0000 7f8906935640 1 -- 172.21.15.132:0/4056211493 >> 172.21.15.132:0/4056211493 conn(0x7f890006d2b0 msgr2=0x7f890006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:55.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.282+0000 7f8906935640 1 -- 172.21.15.132:0/4056211493 shutdown_connections 2024-03-13T07:34:55.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.282+0000 7f8906935640 1 -- 172.21.15.132:0/4056211493 wait complete. 2024-03-13T07:34:55.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.283+0000 7f8906935640 10 monclient: build_initial_monmap 2024-03-13T07:34:55.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.283+0000 7f8906935640 10 monclient: monmap: 2024-03-13T07:34:55.287 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:55.287 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:55.287 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:55.287 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:55.287 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:55.287 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:55.288 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:55.288 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:55.288 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:55.288 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:55.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.284+0000 7f8906935640 1 librados: starting msgr at 2024-03-13T07:34:55.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.284+0000 7f8906935640 1 librados: starting objecter 2024-03-13T07:34:55.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.284+0000 7f8906935640 1 Processor -- start 2024-03-13T07:34:55.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.284+0000 7f8906935640 1 -- start start 2024-03-13T07:34:55.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.284+0000 7f8906935640 1 librados: setting wanted keys 2024-03-13T07:34:55.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.284+0000 7f8906935640 1 librados: calling monclient init 2024-03-13T07:34:55.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.284+0000 7f8906935640 10 monclient: init 2024-03-13T07:34:55.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.285+0000 7f8906935640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:55.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.285+0000 7f8906935640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T07:34:55.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.285+0000 7f8906935640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88e8003a00 0x7f89001db190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:55.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.285+0000 7f8906935640 10 monclient(hunting): picked mon.noname-a con 0x7f88e8003a00 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:55.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.285+0000 7f8906935640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89001393d0 0x7f8900139830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:55.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.285+0000 7f8906935640 10 monclient(hunting): picked mon.noname-c con 0x7f89001393d0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:55.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.285+0000 7f8906935640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89000a9560 0x7f89000a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:55.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.285+0000 7f8906935640 10 monclient(hunting): picked mon.noname-b con 0x7f89000a9560 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:55.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.285+0000 7f8906935640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:55.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.285+0000 7f8906935640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89000b4800 con 0x7f88e8003a00 2024-03-13T07:34:55.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.285+0000 7f8906935640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:55.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.285+0000 7f8906935640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89000afd00 con 0x7f89000a9560 2024-03-13T07:34:55.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.285+0000 7f8906935640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:55.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.285+0000 7f8906935640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89000b4980 con 0x7f89001393d0 2024-03-13T07:34:55.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.285+0000 7f8906935640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:55.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.285+0000 7f8906935640 10 monclient(hunting): authenticate will time out at 1322.693688s 2024-03-13T07:34:55.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.286+0000 7f88ff7fe640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89001393d0 0x7f8900139830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:55.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.286+0000 7f88ff7fe640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89001393d0 0x7f8900139830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:42864/0 (socket says 172.21.15.132:42864) 2024-03-13T07:34:55.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.286+0000 7f88ff7fe640 1 -- 172.21.15.132:0/1177360318 learned_addr learned my addr 172.21.15.132:0/1177360318 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:55.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.286+0000 7f88ff7fe640 10 monclient(hunting): get_auth_request con 0x7f89001393d0 auth_method 0 2024-03-13T07:34:55.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.286+0000 7f88ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:55.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.286+0000 7f88ff7fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:55.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.286+0000 7f88ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:55.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.287+0000 7f88ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:55.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.287+0000 7f88ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:55.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.287+0000 7f88ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:55.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.290+0000 7f88ff7fe640 10 monclient(hunting): handle_auth_done global_id 4310 payload 824 2024-03-13T07:34:55.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.290+0000 7f88ff7fe640 1 -- 172.21.15.132:0/1177360318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89000a9560 msgr2=0x7f89000a99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.290+0000 7f88ff7fe640 1 --2- 172.21.15.132:0/1177360318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89000a9560 0x7f89000a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.290+0000 7f88ff7fe640 1 -- 172.21.15.132:0/1177360318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88e8003a00 msgr2=0x7f89001db190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.290+0000 7f88ff7fe640 1 --2- 172.21.15.132:0/1177360318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88e8003a00 0x7f89001db190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.290+0000 7f88ff7fe640 10 monclient: _finish_hunting 0 2024-03-13T07:34:55.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.290+0000 7f88ff7fe640 1 monclient: found mon.noname-c 2024-03-13T07:34:55.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.290+0000 7f88ff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:55.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.290+0000 7f88ff7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:34:55.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.290+0000 7f88ff7fe640 1 -- 172.21.15.132:0/1177360318 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89000ab090 con 0x7f89001393d0 2024-03-13T07:34:55.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.293+0000 7f88ff7fe640 1 --2- 172.21.15.132:0/1177360318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89001393d0 0x7f8900139830 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f88f400b500 tx=0x7f88f400b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:55.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.293+0000 7f88fcff9640 10 client.?.objecter ms_handle_connect 0x7f89001393d0 2024-03-13T07:34:55.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.293+0000 7f88fcff9640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:55.297 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f88fcff9640 1 -- 172.21.15.132:0/1177360318 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4014070 con 0x7f89001393d0 2024-03-13T07:34:55.297 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f88fcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:55.297 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f88fcff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:55.297 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f88fcff9640 10 monclient: dump: 2024-03-13T07:34:55.297 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:55.297 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:55.297 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:55.297 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:55.297 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:55.297 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:55.297 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:55.298 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:55.298 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:55.298 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:55.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f88fcff9640 10 monclient: _finish_auth 0 2024-03-13T07:34:55.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f88fcff9640 10 monclient: _check_auth_tickets 2024-03-13T07:34:55.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f88fcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:55.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f88fcff9640 1 -- 172.21.15.132:0/1177360318 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f4010870 con 0x7f89001393d0 2024-03-13T07:34:55.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f88fcff9640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:55.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f88fcff9640 1 -- 172.21.15.132:0/1177360318 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4010c60 con 0x7f89001393d0 2024-03-13T07:34:55.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f88fcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:55.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f88fcff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:55.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f88fcff9640 10 monclient: dump: 2024-03-13T07:34:55.298 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:55.298 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:55.299 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:55.299 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:55.299 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:55.299 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:55.299 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:55.299 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:55.299 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:55.299 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:55.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f8906935640 5 monclient: authenticate success, global_id 4310 2024-03-13T07:34:55.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f8906935640 10 monclient: _renew_subs 2024-03-13T07:34:55.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f8906935640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:55.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f88fe7fc640 4 set_mon_vals no callback set 2024-03-13T07:34:55.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f8906935640 1 -- 172.21.15.132:0/1177360318 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89000ab370 con 0x7f89001393d0 2024-03-13T07:34:55.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f8906935640 10 client.4310.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:55.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f8906935640 10 monclient: _renew_subs 2024-03-13T07:34:55.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f8906935640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:55.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f8906935640 1 -- 172.21.15.132:0/1177360318 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89001dbb40 con 0x7f89001393d0 2024-03-13T07:34:55.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.297+0000 7f8906935640 1 librados: init done 2024-03-13T07:34:55.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.298+0000 7f8906935640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:55.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.298+0000 7f8906935640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:55.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.298+0000 7f8906935640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:55.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.298+0000 7f8906935640 1 -- 172.21.15.132:0/1177360318 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8900140190 con 0x7f89001393d0 2024-03-13T07:34:55.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.301+0000 7f3e2effd640 1 -- 172.21.15.132:0/1796523488 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3e30058d50 con 0x7f3e4006f950 2024-03-13T07:34:55.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.301+0000 7f3e2effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2024-03-13T07:34:55.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.301+0000 7f3e2effd640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:55.301 INFO:teuthology.orchestra.run.smithi132.stdout:64424509446 2024-03-13T07:34:55.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.303+0000 7f3e44ec1640 10 librados: watch_flush enter 2024-03-13T07:34:55.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.303+0000 7f3e44ec1640 10 librados: watch_flush exit 2024-03-13T07:34:55.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.303+0000 7f3e44ec1640 20 client.4308.objecter shutdown clearing up homeless session... 2024-03-13T07:34:55.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.303+0000 7f3e44ec1640 10 client.4308.objecter successfully canceled tick 2024-03-13T07:34:55.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.303+0000 7f3e44ec1640 1 -- 172.21.15.132:0/1796523488 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3e2006d560 msgr2=0x7f3e2006fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.303+0000 7f3e44ec1640 1 --2- 172.21.15.132:0/1796523488 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3e2006d560 0x7f3e2006fa20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e380020a0 tx=0x7f3e38001690 comp rx=0 tx=0).stop 2024-03-13T07:34:55.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.303+0000 7f3e44ec1640 10 monclient: shutdown 2024-03-13T07:34:55.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.303+0000 7f3e44ec1640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:55.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.303+0000 7f3e44ec1640 1 -- 172.21.15.132:0/1796523488 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e4006f950 msgr2=0x7f3e40147d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.303+0000 7f3e44ec1640 1 --2- 172.21.15.132:0/1796523488 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e4006f950 0x7f3e40147d10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000c450 tx=0x7f3e3000c910 comp rx=0 tx=0).stop 2024-03-13T07:34:55.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.304+0000 7f3e44ec1640 1 -- 172.21.15.132:0/1796523488 shutdown_connections 2024-03-13T07:34:55.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.304+0000 7f3e44ec1640 1 --2- 172.21.15.132:0/1796523488 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e40144340 0x7f3e40148250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.304+0000 7f3e44ec1640 1 --2- 172.21.15.132:0/1796523488 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3e2006d560 0x7f3e2006fa20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.304+0000 7f3e44ec1640 1 --2- 172.21.15.132:0/1796523488 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e4006f950 0x7f3e40147d10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.304+0000 7f3e44ec1640 1 --2- 172.21.15.132:0/1796523488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e4006e610 0x7f3e400b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.304+0000 7f3e44ec1640 1 -- 172.21.15.132:0/1796523488 >> 172.21.15.132:0/1796523488 conn(0x7f3e400b3e20 msgr2=0x7f3e400b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:55.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.304+0000 7f3e44ec1640 1 -- 172.21.15.132:0/1796523488 shutdown_connections 2024-03-13T07:34:55.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.304+0000 7f3e44ec1640 1 -- 172.21.15.132:0/1796523488 wait complete. 2024-03-13T07:34:55.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.304+0000 7f3e44ec1640 1 librados: shutdown 2024-03-13T07:34:55.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2024-03-13T07:34:55.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.416+0000 7f2d50ec2640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-03-13T07:34:55.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.416+0000 7f2d50ec2640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-03-13T07:34:55.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.416+0000 7f2d50ec2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:55.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.416+0000 7f2d50ec2640 1 -- 172.21.15.132:0/4088929904 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2d4c1427b0 con 0x7f2d4c0b5320 2024-03-13T07:34:55.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.460+0000 7fb33cff9640 1 -- 172.21.15.132:0/2065159372 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fb33801c070 con 0x7fb340140400 2024-03-13T07:34:55.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.460+0000 7fb33cff9640 1 --2- 172.21.15.132:0/2065159372 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb32c06d5b0 0x7fb32c06fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:55.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.461+0000 7fb33ffff640 1 --2- 172.21.15.132:0/2065159372 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb32c06d5b0 0x7fb32c06fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:55.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.461+0000 7fb33ffff640 10 monclient: get_auth_request con 0x7fb32c06d5b0 auth_method 0 2024-03-13T07:34:55.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.461+0000 7fb33cff9640 10 client.4327.objecter ms_handle_connect 0x7fb32c06d5b0 2024-03-13T07:34:55.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.461+0000 7fb33ffff640 1 --2- 172.21.15.132:0/2065159372 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb32c06d5b0 0x7fb32c06fa70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb330036000 tx=0x7fb330030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:55.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.463+0000 7fb33cff9640 1 -- 172.21.15.132:0/2065159372 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb338057180 con 0x7fb340140400 2024-03-13T07:34:55.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.463+0000 7fb33cff9640 10 client.4327.objecter ms_dispatch 0x7fb340000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:55.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.463+0000 7fb33cff9640 3 client.4327.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:55.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.463+0000 7fb33cff9640 3 client.4327.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:55.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.463+0000 7fb33cff9640 20 client.4327.objecter dump_active .. 0 homeless 2024-03-13T07:34:55.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.473+0000 7fb33cff9640 1 -- 172.21.15.132:0/2065159372 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fb338023020 con 0x7fb340140400 2024-03-13T07:34:55.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.473+0000 7fb33cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:55.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.473+0000 7fb33cff9640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:55.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f0ff9640 1 -- 172.21.15.132:0/1573811828 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4015070 con 0x7f05e0004110 2024-03-13T07:34:55.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:55.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f0ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:55.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f0ff9640 10 monclient: dump: 2024-03-13T07:34:55.474 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:55.474 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:55.474 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:55.474 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:55.474 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:55.474 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:55.475 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:55.475 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:55.475 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:55.475 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:55.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f0ff9640 10 monclient: _finish_auth 0 2024-03-13T07:34:55.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f0ff9640 10 monclient: _check_auth_tickets 2024-03-13T07:34:55.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:55.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f0ff9640 1 -- 172.21.15.132:0/1573811828 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e4004220 con 0x7f05e0004110 2024-03-13T07:34:55.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f0ff9640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:55.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f9974640 5 monclient: authenticate success, global_id 4324 2024-03-13T07:34:55.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f9974640 10 monclient: _renew_subs 2024-03-13T07:34:55.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f9974640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:55.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f9974640 1 -- 172.21.15.132:0/1573811828 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f40a9950 con 0x7f05e0004110 2024-03-13T07:34:55.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f1ffb640 4 set_mon_vals no callback set 2024-03-13T07:34:55.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f9974640 10 client.4324.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:55.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f9974640 10 monclient: _renew_subs 2024-03-13T07:34:55.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f9974640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:55.477 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f9974640 1 -- 172.21.15.132:0/1573811828 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f41d8810 con 0x7f05e0004110 2024-03-13T07:34:55.477 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f9974640 1 librados: init done 2024-03-13T07:34:55.477 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f0ff9640 1 -- 172.21.15.132:0/1573811828 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4007620 con 0x7f05e0004110 2024-03-13T07:34:55.477 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:55.477 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:55.477 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.474+0000 7f05f0ff9640 10 monclient: dump: 2024-03-13T07:34:55.477 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:55.477 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:55.477 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:55.477 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:55.477 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:55.477 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:55.477 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:55.477 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:55.477 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:55.478 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:55.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.475+0000 7f05f9974640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:55.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.476+0000 7f05f9974640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:55.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.476+0000 7f05f9974640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:55.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.476+0000 7f05f9974640 1 -- 172.21.15.132:0/1573811828 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f40b26c0 con 0x7f05e0004110 2024-03-13T07:34:55.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.485+0000 7f42bc521640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-03-13T07:34:55.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.486+0000 7f42bc521640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-03-13T07:34:55.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.486+0000 7f42bc521640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:55.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.486+0000 7f42bc521640 1 -- 172.21.15.132:0/547973600 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f42b404f000 con 0x7f42b40b5320 2024-03-13T07:34:55.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.491+0000 7f42aaffd640 1 -- 172.21.15.132:0/547973600 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f42ac023020 con 0x7f42b40b5320 2024-03-13T07:34:55.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.491+0000 7f42aaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-03-13T07:34:55.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.491+0000 7f42aaffd640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:55.491 INFO:teuthology.orchestra.run.smithi132.stdout:85899345926 2024-03-13T07:34:55.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 10 librados: watch_flush enter 2024-03-13T07:34:55.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 10 librados: watch_flush exit 2024-03-13T07:34:55.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 20 client.4317.objecter shutdown clearing up homeless session... 2024-03-13T07:34:55.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 10 client.4317.objecter successfully canceled tick 2024-03-13T07:34:55.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 1 -- 172.21.15.132:0/547973600 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f429806d5b0 msgr2=0x7f429806fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 1 --2- 172.21.15.132:0/547973600 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f429806d5b0 0x7f429806fa70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f42a4001560 tx=0x7f42a4007990 comp rx=0 tx=0).stop 2024-03-13T07:34:55.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 10 monclient: shutdown 2024-03-13T07:34:55.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:55.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 1 -- 172.21.15.132:0/547973600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42b40b5320 msgr2=0x7f42b41d8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 1 --2- 172.21.15.132:0/547973600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42b40b5320 0x7f42b41d8300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00ba50 tx=0x7f42ac00bf10 comp rx=0 tx=0).stop 2024-03-13T07:34:55.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 1 -- 172.21.15.132:0/547973600 shutdown_connections 2024-03-13T07:34:55.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 1 --2- 172.21.15.132:0/547973600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42b413fb70 0x7f42b41d8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 1 --2- 172.21.15.132:0/547973600 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f429806d5b0 0x7f429806fa70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 1 --2- 172.21.15.132:0/547973600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42b40b5320 0x7f42b41d8300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 1 --2- 172.21.15.132:0/547973600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42b40b0220 0x7f42b41cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 1 -- 172.21.15.132:0/547973600 >> 172.21.15.132:0/547973600 conn(0x7f42b406d2b0 msgr2=0x7f42b413c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:55.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 1 -- 172.21.15.132:0/547973600 shutdown_connections 2024-03-13T07:34:55.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 1 -- 172.21.15.132:0/547973600 wait complete. 2024-03-13T07:34:55.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.493+0000 7f42bc521640 1 librados: shutdown 2024-03-13T07:34:55.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 85899345927 got 85899345926 for osd.2 2024-03-13T07:34:55.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.507+0000 7f05f0ff9640 1 -- 172.21.15.132:0/1573811828 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f05e40079d0 con 0x7f05e0004110 2024-03-13T07:34:55.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.507+0000 7f05f0ff9640 1 --2- 172.21.15.132:0/1573811828 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f05d406d5b0 0x7f05d406fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:55.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.507+0000 7f05ebfff640 1 --2- 172.21.15.132:0/1573811828 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f05d406d5b0 0x7f05d406fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:55.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.507+0000 7f05ebfff640 10 monclient: get_auth_request con 0x7f05d406d5b0 auth_method 0 2024-03-13T07:34:55.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.508+0000 7f05f0ff9640 10 client.4324.objecter ms_handle_connect 0x7f05d406d5b0 2024-03-13T07:34:55.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.508+0000 7f05ebfff640 1 --2- 172.21.15.132:0/1573811828 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f05d406d5b0 0x7f05d406fa70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f05dc034000 tx=0x7f05dc039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:55.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.512+0000 7f05f0ff9640 1 -- 172.21.15.132:0/1573811828 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f05e4057420 con 0x7f05e0004110 2024-03-13T07:34:55.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.512+0000 7f05f0ff9640 10 client.4324.objecter ms_dispatch 0x7f05f4000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:55.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.512+0000 7f05f0ff9640 3 client.4324.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:55.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.512+0000 7f05f0ff9640 3 client.4324.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:55.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.512+0000 7f05f0ff9640 20 client.4324.objecter dump_active .. 0 homeless 2024-03-13T07:34:55.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.573+0000 7f88fcff9640 1 -- 172.21.15.132:0/1177360318 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f88f4017070 con 0x7f89001393d0 2024-03-13T07:34:55.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.573+0000 7f88fcff9640 1 --2- 172.21.15.132:0/1177360318 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f88ec06d5b0 0x7f88ec06fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:55.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.574+0000 7f88fffff640 1 --2- 172.21.15.132:0/1177360318 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f88ec06d5b0 0x7f88ec06fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:55.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.574+0000 7f88fffff640 10 monclient: get_auth_request con 0x7f88ec06d5b0 auth_method 0 2024-03-13T07:34:55.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.574+0000 7f88fcff9640 10 client.4310.objecter ms_handle_connect 0x7f88ec06d5b0 2024-03-13T07:34:55.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.574+0000 7f88fffff640 1 --2- 172.21.15.132:0/1177360318 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f88ec06d5b0 0x7f88ec06fa70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f88f0034000 tx=0x7f88f0007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:55.589 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.589+0000 7f88fcff9640 1 -- 172.21.15.132:0/1177360318 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f88f40572d0 con 0x7f89001393d0 2024-03-13T07:34:55.589 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.589+0000 7f88fcff9640 10 client.4310.objecter ms_dispatch 0x7f8900000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:55.589 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.589+0000 7f88fcff9640 3 client.4310.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:55.590 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.589+0000 7f88fcff9640 3 client.4310.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:55.590 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.589+0000 7f88fcff9640 20 client.4310.objecter dump_active .. 0 homeless 2024-03-13T07:34:55.590 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.589+0000 7f88fcff9640 1 -- 172.21.15.132:0/1177360318 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f8900140190 con 0x7f89001393d0 2024-03-13T07:34:55.590 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.589+0000 7f88fcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:55.590 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.589+0000 7f88fcff9640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:55.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.672+0000 7fb34699a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-03-13T07:34:55.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.672+0000 7fb34699a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-03-13T07:34:55.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.672+0000 7fb34699a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:55.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.672+0000 7fb34699a640 1 -- 172.21.15.132:0/2065159372 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb340067e10 con 0x7fb340140400 2024-03-13T07:34:55.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.770+0000 7f8906935640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2024-03-13T07:34:55.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.770+0000 7f8906935640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2024-03-13T07:34:55.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.770+0000 7f8906935640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:55.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.770+0000 7f8906935640 1 -- 172.21.15.132:0/1177360318 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f890004f000 con 0x7f89001393d0 2024-03-13T07:34:55.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.774+0000 7f88fcff9640 1 -- 172.21.15.132:0/1177360318 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f88f4059930 con 0x7f89001393d0 2024-03-13T07:34:55.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.774+0000 7f88fcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2024-03-13T07:34:55.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.774+0000 7f88fcff9640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:55.774 INFO:teuthology.orchestra.run.smithi132.stdout:55834574852 2024-03-13T07:34:55.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 10 librados: watch_flush enter 2024-03-13T07:34:55.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 10 librados: watch_flush exit 2024-03-13T07:34:55.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 20 client.4310.objecter shutdown clearing up homeless session... 2024-03-13T07:34:55.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 10 client.4310.objecter successfully canceled tick 2024-03-13T07:34:55.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 1 -- 172.21.15.132:0/1177360318 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f88ec06d5b0 msgr2=0x7f88ec06fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 1 --2- 172.21.15.132:0/1177360318 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f88ec06d5b0 0x7f88ec06fa70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f88f0034000 tx=0x7f88f0007000 comp rx=0 tx=0).stop 2024-03-13T07:34:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 10 monclient: shutdown 2024-03-13T07:34:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 1 -- 172.21.15.132:0/1177360318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89001393d0 msgr2=0x7f8900139830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 1 --2- 172.21.15.132:0/1177360318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89001393d0 0x7f8900139830 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f88f400b500 tx=0x7f88f400b9c0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 1 -- 172.21.15.132:0/1177360318 shutdown_connections 2024-03-13T07:34:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 1 --2- 172.21.15.132:0/1177360318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89000a9560 0x7f89000a99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 1 --2- 172.21.15.132:0/1177360318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89001393d0 0x7f8900139830 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 1 --2- 172.21.15.132:0/1177360318 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f88ec06d5b0 0x7f88ec06fa70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 1 --2- 172.21.15.132:0/1177360318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88e8003a00 0x7f89001db190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 1 -- 172.21.15.132:0/1177360318 >> 172.21.15.132:0/1177360318 conn(0x7f890006d2b0 msgr2=0x7f89000b6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:55.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f05f0ff9640 1 -- 172.21.15.132:0/1573811828 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f05f40b26c0 con 0x7f05e0004110 2024-03-13T07:34:55.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f05f0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:55.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f05f0ff9640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:55.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 1 -- 172.21.15.132:0/1177360318 shutdown_connections 2024-03-13T07:34:55.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 1 -- 172.21.15.132:0/1177360318 wait complete. 2024-03-13T07:34:55.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.776+0000 7f8906935640 1 librados: shutdown 2024-03-13T07:34:55.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.777+0000 7f2d42ffd640 1 -- 172.21.15.132:0/4088929904 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d34058e70 con 0x7f2d4c0b5320 2024-03-13T07:34:55.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.777+0000 7f2d42ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-03-13T07:34:55.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.777+0000 7f2d42ffd640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:55.779 INFO:teuthology.orchestra.run.smithi132.stdout:68719476741 2024-03-13T07:34:55.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.779+0000 7f2d50ec2640 10 librados: watch_flush enter 2024-03-13T07:34:55.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.779+0000 7f2d50ec2640 10 librados: watch_flush exit 2024-03-13T07:34:55.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.779+0000 7f2d50ec2640 20 client.4321.objecter shutdown clearing up homeless session... 2024-03-13T07:34:55.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.779+0000 7f2d50ec2640 10 client.4321.objecter successfully canceled tick 2024-03-13T07:34:55.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.779+0000 7f2d50ec2640 1 -- 172.21.15.132:0/4088929904 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2d2c06d5b0 msgr2=0x7f2d2c06fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.779+0000 7f2d50ec2640 1 --2- 172.21.15.132:0/4088929904 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2d2c06d5b0 0x7f2d2c06fa70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c035000 tx=0x7f2d3c030000 comp rx=0 tx=0).stop 2024-03-13T07:34:55.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.779+0000 7f2d50ec2640 10 monclient: shutdown 2024-03-13T07:34:55.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.779+0000 7f2d50ec2640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:55.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.779+0000 7f2d50ec2640 1 -- 172.21.15.132:0/4088929904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d4c0b5320 msgr2=0x7f2d4c1403e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.779+0000 7f2d50ec2640 1 --2- 172.21.15.132:0/4088929904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d4c0b5320 0x7f2d4c1403e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400c410 tx=0x7f2d3400c8d0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.780+0000 7f2d50ec2640 1 -- 172.21.15.132:0/4088929904 shutdown_connections 2024-03-13T07:34:55.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.780+0000 7f2d50ec2640 1 --2- 172.21.15.132:0/4088929904 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2d2c06d5b0 0x7f2d2c06fa70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.780+0000 7f2d50ec2640 1 --2- 172.21.15.132:0/4088929904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d4c13fd40 0x7f2d4c141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.780+0000 7f2d50ec2640 1 --2- 172.21.15.132:0/4088929904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d4c0b5320 0x7f2d4c1403e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.780+0000 7f2d50ec2640 1 --2- 172.21.15.132:0/4088929904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d4c0b1ac0 0x7f2d4c138530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.780+0000 7f2d50ec2640 1 -- 172.21.15.132:0/4088929904 >> 172.21.15.132:0/4088929904 conn(0x7f2d4c06d2b0 msgr2=0x7f2d4c13f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:55.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.780+0000 7f2d50ec2640 1 -- 172.21.15.132:0/4088929904 shutdown_connections 2024-03-13T07:34:55.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.780+0000 7f2d50ec2640 1 -- 172.21.15.132:0/4088929904 wait complete. 2024-03-13T07:34:55.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.780+0000 7f2d50ec2640 1 librados: shutdown 2024-03-13T07:34:55.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.786+0000 7fb33cff9640 1 -- 172.21.15.132:0/2065159372 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb3380597e0 con 0x7fb340140400 2024-03-13T07:34:55.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.786+0000 7fb33cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-03-13T07:34:55.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.786+0000 7fb33cff9640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:55.786 INFO:teuthology.orchestra.run.smithi132.stdout:85899345928 2024-03-13T07:34:55.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.788+0000 7fb34699a640 10 librados: watch_flush enter 2024-03-13T07:34:55.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.788+0000 7fb34699a640 10 librados: watch_flush exit 2024-03-13T07:34:55.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.788+0000 7fb34699a640 20 client.4327.objecter shutdown clearing up homeless session... 2024-03-13T07:34:55.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.789+0000 7fb34699a640 10 client.4327.objecter successfully canceled tick 2024-03-13T07:34:55.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.789+0000 7fb34699a640 1 -- 172.21.15.132:0/2065159372 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb32c06d5b0 msgr2=0x7fb32c06fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.789+0000 7fb34699a640 1 --2- 172.21.15.132:0/2065159372 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb32c06d5b0 0x7fb32c06fa70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb330036000 tx=0x7fb330030000 comp rx=0 tx=0).stop 2024-03-13T07:34:55.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.789+0000 7fb34699a640 10 monclient: shutdown 2024-03-13T07:34:55.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.789+0000 7fb34699a640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:55.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.789+0000 7fb34699a640 1 -- 172.21.15.132:0/2065159372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb340140400 msgr2=0x7fb3400ab5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:55.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.789+0000 7fb34699a640 1 --2- 172.21.15.132:0/2065159372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb340140400 0x7fb3400ab5e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb33800b990 tx=0x7fb33800be50 comp rx=0 tx=0).stop 2024-03-13T07:34:55.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.789+0000 7fb34699a640 1 -- 172.21.15.132:0/2065159372 shutdown_connections 2024-03-13T07:34:55.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.789+0000 7fb34699a640 1 --2- 172.21.15.132:0/2065159372 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb340140400 0x7fb3400ab5e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.789+0000 7fb34699a640 1 --2- 172.21.15.132:0/2065159372 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fb32c06d5b0 0x7fb32c06fa70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.789+0000 7fb34699a640 1 --2- 172.21.15.132:0/2065159372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3400b1e70 0x7fb3400aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.789+0000 7fb34699a640 1 --2- 172.21.15.132:0/2065159372 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3400b00f0 0x7fb34013e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:55.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.789+0000 7fb34699a640 1 -- 172.21.15.132:0/2065159372 >> 172.21.15.132:0/2065159372 conn(0x7fb3400b80e0 msgr2=0x7fb3400b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:55.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.789+0000 7fb34699a640 1 -- 172.21.15.132:0/2065159372 shutdown_connections 2024-03-13T07:34:55.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.789+0000 7fb34699a640 1 -- 172.21.15.132:0/2065159372 wait complete. 2024-03-13T07:34:55.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.789+0000 7fb34699a640 1 librados: shutdown 2024-03-13T07:34:55.792 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.7 2024-03-13T07:34:55.799 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476741 for osd.3 2024-03-13T07:34:55.802 INFO:tasks.ceph.ceph_manager.ceph:need seq 85899345929 got 85899345928 for osd.1 2024-03-13T07:34:55.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.980+0000 7f05f9974640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-03-13T07:34:55.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.980+0000 7f05f9974640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-03-13T07:34:55.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.980+0000 7f05f9974640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:55.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:55.980+0000 7f05f9974640 1 -- 172.21.15.132:0/1573811828 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f05f404f000 con 0x7f05e0004110 2024-03-13T07:34:56.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.012+0000 7f05f0ff9640 1 -- 172.21.15.132:0/1573811828 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f05e4059a80 con 0x7f05e0004110 2024-03-13T07:34:56.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.012+0000 7f05f0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-03-13T07:34:56.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.012+0000 7f05f0ff9640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:56.013 INFO:teuthology.orchestra.run.smithi132.stdout:85899345928 2024-03-13T07:34:56.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 10 librados: watch_flush enter 2024-03-13T07:34:56.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 10 librados: watch_flush exit 2024-03-13T07:34:56.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 20 client.4324.objecter shutdown clearing up homeless session... 2024-03-13T07:34:56.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 10 client.4324.objecter successfully canceled tick 2024-03-13T07:34:56.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 1 -- 172.21.15.132:0/1573811828 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f05d406d5b0 msgr2=0x7f05d406fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 1 --2- 172.21.15.132:0/1573811828 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f05d406d5b0 0x7f05d406fa70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f05dc034000 tx=0x7f05dc039040 comp rx=0 tx=0).stop 2024-03-13T07:34:56.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 10 monclient: shutdown 2024-03-13T07:34:56.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:56.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 1 -- 172.21.15.132:0/1573811828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05e0004110 msgr2=0x7f05f41403e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 1 --2- 172.21.15.132:0/1573811828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05e0004110 0x7f05f41403e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f05e400c7f0 tx=0x7f05e400ccb0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 1 -- 172.21.15.132:0/1573811828 shutdown_connections 2024-03-13T07:34:56.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 1 --2- 172.21.15.132:0/1573811828 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05f4138840 0x7f05f41d82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 1 --2- 172.21.15.132:0/1573811828 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f05d406d5b0 0x7f05d406fa70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 1 --2- 172.21.15.132:0/1573811828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05f40a94d0 0x7f05f4138300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 1 --2- 172.21.15.132:0/1573811828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05e0004110 0x7f05f41403e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 1 -- 172.21.15.132:0/1573811828 >> 172.21.15.132:0/1573811828 conn(0x7f05f406d2b0 msgr2=0x7f05f40b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:56.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 1 -- 172.21.15.132:0/1573811828 shutdown_connections 2024-03-13T07:34:56.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 1 -- 172.21.15.132:0/1573811828 wait complete. 2024-03-13T07:34:56.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.014+0000 7f05f9974640 1 librados: shutdown 2024-03-13T07:34:56.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 85899345928 got 85899345928 for osd.0 2024-03-13T07:34:56.026 DEBUG:teuthology.parallel:result is None 2024-03-13T07:34:56.134 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-13T07:34:56.223 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-13T07:34:56.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.237+0000 7f895b45d640 10 monclient: get_monmap_and_config 2024-03-13T07:34:56.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.237+0000 7f895b45d640 10 monclient: build_initial_monmap 2024-03-13T07:34:56.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.237+0000 7f895b45d640 10 monclient: monmap: 2024-03-13T07:34:56.238 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:56.238 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.238 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:56.238480+0000 2024-03-13T07:34:56.238 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:56.238480+0000 2024-03-13T07:34:56.238 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:56.238 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.238 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:56.238 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:56.238 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:56.239 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:56.239 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:56.239 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.238+0000 7f895b45d640 1 Processor -- start 2024-03-13T07:34:56.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.238+0000 7f895b45d640 1 -- start start 2024-03-13T07:34:56.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.238+0000 7f895b45d640 10 monclient: init 2024-03-13T07:34:56.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.239+0000 7f895b45d640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:56.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.239+0000 7f895b45d640 10 monclient: _add_conns ranks=[0,3,2,1,4] 2024-03-13T07:34:56.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.239+0000 7f895b45d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f89540b5320 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:56.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.239+0000 7f895b45d640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f89540b0220 0x7f895413f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.239+0000 7f895b45d640 10 monclient(hunting): picked mon.noname-a con 0x7f89540b0220 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:56.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.239+0000 7f895b45d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895413fb70 0x7f895414cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.239+0000 7f895b45d640 10 monclient(hunting): picked mon.noname-b con 0x7f895413fb70 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:56.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.239+0000 7f895b45d640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89540b1150 con 0x7f89540b5320 2024-03-13T07:34:56.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.239+0000 7f895b45d640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.239+0000 7f895b45d640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89540b0690 con 0x7f89540b0220 2024-03-13T07:34:56.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.239+0000 7f895b45d640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.239+0000 7f895b45d640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895413ffa0 con 0x7f895413fb70 2024-03-13T07:34:56.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.239+0000 7f895b45d640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:56.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.239+0000 7f895b45d640 10 monclient(hunting): authenticate will time out at 1323.647248s 2024-03-13T07:34:56.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.239+0000 7f89589f8640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f89540b0220 0x7f895413f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.239+0000 7f89599fa640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895413fb70 0x7f895414cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.240+0000 7f89589f8640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f89540b0220 0x7f895413f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34552/0 (socket says 172.21.15.132:34552) 2024-03-13T07:34:56.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.240+0000 7f89589f8640 1 -- 172.21.15.132:0/2814665532 learned_addr learned my addr 172.21.15.132:0/2814665532 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:56.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.240+0000 7f89589f8640 10 monclient(hunting): get_auth_request con 0x7f89540b0220 auth_method 0 2024-03-13T07:34:56.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.240+0000 7f89589f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.240+0000 7f89589f8640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.240+0000 7f89589f8640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.240+0000 7f89599fa640 10 monclient(hunting): get_auth_request con 0x7f895413fb70 auth_method 0 2024-03-13T07:34:56.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.240+0000 7f89599fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.240+0000 7f89599fa640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.240+0000 7f89599fa640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.240+0000 7f89599fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.240+0000 7f89599fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.240+0000 7f89599fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.241+0000 7f89589f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.241+0000 7f89589f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.241+0000 7f89589f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.242+0000 7f89599fa640 10 monclient(hunting): handle_auth_done global_id 4313 payload 274 2024-03-13T07:34:56.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.242+0000 7f89599fa640 1 -- 172.21.15.132:0/2814665532 >> v2:172.21.15.132:3300/0 conn(0x7f89540b0220 msgr2=0x7f895413f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:56.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.242+0000 7f89599fa640 1 --2- 172.21.15.132:0/2814665532 >> v2:172.21.15.132:3300/0 conn(0x7f89540b0220 0x7f895413f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.242+0000 7f89599fa640 1 -- 172.21.15.132:0/2814665532 >> v1:172.21.15.132:6789/0 conn(0x7f89540b5320 legacy=0x7f89540b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.242+0000 7f89599fa640 10 monclient: _finish_hunting 0 2024-03-13T07:34:56.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.242+0000 7f89599fa640 1 monclient: found mon.noname-b 2024-03-13T07:34:56.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.242+0000 7f89599fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:56.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.242+0000 7f89599fa640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T07:34:56.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.242+0000 7f89599fa640 1 -- 172.21.15.132:0/2814665532 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89540b13c0 con 0x7f895413fb70 2024-03-13T07:34:56.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.244+0000 7f89599fa640 1 --2- 172.21.15.132:0/2814665532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895413fb70 0x7f895414cef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f894c009d90 tx=0x7f894c02bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d884d8bc91528a92 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:56.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f894bfff640 1 -- 172.21.15.132:0/2814665532 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c039070 con 0x7f895413fb70 2024-03-13T07:34:56.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f894bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f894bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:56.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f894bfff640 10 monclient: dump: 2024-03-13T07:34:56.248 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.248 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.248 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.248 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.248 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.248 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.249 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.249 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.249 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.249 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f894bfff640 10 monclient: _finish_auth 0 2024-03-13T07:34:56.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f894bfff640 10 monclient: _check_auth_tickets 2024-03-13T07:34:56.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f894bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:56.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f894bfff640 1 -- 172.21.15.132:0/2814665532 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c02cb30 con 0x7f895413fb70 2024-03-13T07:34:56.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f895b45d640 5 monclient: authenticate success, global_id 4313 2024-03-13T07:34:56.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f895b45d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:56.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f894bfff640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:56.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f895b45d640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:56.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f894bfff640 1 -- 172.21.15.132:0/2814665532 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c034440 con 0x7f895413fb70 2024-03-13T07:34:56.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f895b45d640 4 set_mon_vals no callback set 2024-03-13T07:34:56.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f894bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f894bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:56.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.248+0000 7f894bfff640 10 monclient: dump: 2024-03-13T07:34:56.250 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.250 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.250 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.250 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.250 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.250 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.250 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.250 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.250 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.250 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.253 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.249+0000 7f895b45d640 10 monclient: shutdown 2024-03-13T07:34:56.253 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.249+0000 7f895b45d640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:56.253 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.249+0000 7f895b45d640 1 -- 172.21.15.132:0/2814665532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895413fb70 msgr2=0x7f895414cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.249+0000 7f895b45d640 1 --2- 172.21.15.132:0/2814665532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895413fb70 0x7f895414cef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f894c009d90 tx=0x7f894c02bd80 comp rx=0 tx=0).stop 2024-03-13T07:34:56.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.249+0000 7f895b45d640 1 -- 172.21.15.132:0/2814665532 shutdown_connections 2024-03-13T07:34:56.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.249+0000 7f895b45d640 1 --2- 172.21.15.132:0/2814665532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895413fb70 0x7f895414cef0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.249+0000 7f895b45d640 1 --2- 172.21.15.132:0/2814665532 >> v2:172.21.15.132:3300/0 conn(0x7f89540b0220 0x7f895413f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.249+0000 7f895b45d640 1 -- 172.21.15.132:0/2814665532 >> 172.21.15.132:0/2814665532 conn(0x7f895406d2b0 msgr2=0x7f895406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:56.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.249+0000 7f895b45d640 1 -- 172.21.15.132:0/2814665532 shutdown_connections 2024-03-13T07:34:56.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.249+0000 7f895b45d640 1 -- 172.21.15.132:0/2814665532 wait complete. 2024-03-13T07:34:56.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.250+0000 7f895b45d640 10 monclient: build_initial_monmap 2024-03-13T07:34:56.254 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.250+0000 7f895b45d640 10 monclient: monmap: 2024-03-13T07:34:56.254 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:56.254 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:56.254 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:56.255 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:56.255 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:56.255 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.255 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:56.255 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:56.255 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:56.255 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.255 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.250+0000 7f895b45d640 1 librados: starting msgr at 2024-03-13T07:34:56.255 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.250+0000 7f895b45d640 1 librados: starting objecter 2024-03-13T07:34:56.255 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.250+0000 7f895b45d640 1 Processor -- start 2024-03-13T07:34:56.255 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.250+0000 7f895b45d640 1 -- start start 2024-03-13T07:34:56.255 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.250+0000 7f895b45d640 1 librados: setting wanted keys 2024-03-13T07:34:56.255 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.250+0000 7f895b45d640 1 librados: calling monclient init 2024-03-13T07:34:56.255 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.250+0000 7f895b45d640 10 monclient: init 2024-03-13T07:34:56.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f895b45d640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:56.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f895b45d640 10 monclient: _add_conns ranks=[2,1,0] 2024-03-13T07:34:56.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f895b45d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89540b0220 0x7f8954145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f895b45d640 10 monclient(hunting): picked mon.noname-c con 0x7f89540b0220 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:56.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f895b45d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89540b5320 0x7f89541403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f895b45d640 10 monclient(hunting): picked mon.noname-b con 0x7f89540b5320 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:56.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f895b45d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f895413fb70 0x7f8954141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f895b45d640 10 monclient(hunting): picked mon.noname-a con 0x7f895413fb70 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:56.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f895b45d640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f895b45d640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89540b4800 con 0x7f895413fb70 2024-03-13T07:34:56.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f895b45d640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f895b45d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89540afd00 con 0x7f89540b5320 2024-03-13T07:34:56.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f895b45d640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f895b45d640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89540b4980 con 0x7f89540b0220 2024-03-13T07:34:56.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f895b45d640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:56.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f895b45d640 10 monclient(hunting): authenticate will time out at 1323.659187s 2024-03-13T07:34:56.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f89599fa640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f895413fb70 0x7f8954141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.251+0000 7f89591f9640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89540b0220 0x7f8954145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89589f8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89540b5320 0x7f89541403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89599fa640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f895413fb70 0x7f8954141e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34568/0 (socket says 172.21.15.132:34568) 2024-03-13T07:34:56.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89591f9640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89540b0220 0x7f8954145d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:57768/0 (socket says 172.21.15.132:57768) 2024-03-13T07:34:56.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89599fa640 1 -- 172.21.15.132:0/1812550876 learned_addr learned my addr 172.21.15.132:0/1812550876 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:56.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89599fa640 10 monclient(hunting): get_auth_request con 0x7f895413fb70 auth_method 0 2024-03-13T07:34:56.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89599fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89599fa640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89599fa640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89591f9640 10 monclient(hunting): get_auth_request con 0x7f89540b0220 auth_method 0 2024-03-13T07:34:56.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89591f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89591f9640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89591f9640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89589f8640 10 monclient(hunting): get_auth_request con 0x7f89540b5320 auth_method 0 2024-03-13T07:34:56.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89589f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89589f8640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89589f8640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89599fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89599fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89599fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89591f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89591f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89591f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89589f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89589f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.252+0000 7f89589f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.261 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.255+0000 7f89591f9640 10 monclient(hunting): handle_auth_done global_id 4316 payload 824 2024-03-13T07:34:56.261 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.255+0000 7f89591f9640 1 -- 172.21.15.132:0/1812550876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89540b5320 msgr2=0x7f89541403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.261 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.255+0000 7f89591f9640 1 --2- 172.21.15.132:0/1812550876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89540b5320 0x7f89541403e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.255+0000 7f89591f9640 1 -- 172.21.15.132:0/1812550876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f895413fb70 msgr2=0x7f8954141e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.255+0000 7f89591f9640 1 --2- 172.21.15.132:0/1812550876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f895413fb70 0x7f8954141e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.255+0000 7f89591f9640 10 monclient: _finish_hunting 0 2024-03-13T07:34:56.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.255+0000 7f89591f9640 1 monclient: found mon.noname-c 2024-03-13T07:34:56.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.255+0000 7f89591f9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:56.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.255+0000 7f89591f9640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:34:56.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.255+0000 7f89591f9640 1 -- 172.21.15.132:0/1812550876 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8954142370 con 0x7f89540b0220 2024-03-13T07:34:56.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.255+0000 7f89599fa640 1 --2- 172.21.15.132:0/1812550876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f895413fb70 0x7f8954141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-13T07:34:56.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.255+0000 7f89589f8640 1 --2- 172.21.15.132:0/1812550876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89540b5320 0x7f89541403e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-13T07:34:56.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.257+0000 7f89591f9640 1 --2- 172.21.15.132:0/1812550876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89540b0220 0x7f8954145d40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f89540b5850 tx=0x7f895000c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:56.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.257+0000 7f8949ffb640 10 client.?.objecter ms_handle_connect 0x7f89540b0220 2024-03-13T07:34:56.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.257+0000 7f8949ffb640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:56.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f8949ffb640 1 -- 172.21.15.132:0/1812550876 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950017070 con 0x7f89540b0220 2024-03-13T07:34:56.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f8949ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f8949ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:56.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f8949ffb640 10 monclient: dump: 2024-03-13T07:34:56.263 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.263 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.263 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.263 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.263 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.263 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.263 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.263 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.263 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.263 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.263 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f8949ffb640 10 monclient: _finish_auth 0 2024-03-13T07:34:56.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f8949ffb640 10 monclient: _check_auth_tickets 2024-03-13T07:34:56.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f8949ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:56.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f8949ffb640 1 -- 172.21.15.132:0/1812550876 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89500052f0 con 0x7f89540b0220 2024-03-13T07:34:56.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f8949ffb640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:56.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f8949ffb640 1 -- 172.21.15.132:0/1812550876 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950003850 con 0x7f89540b0220 2024-03-13T07:34:56.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f8949ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f8949ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:56.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f8949ffb640 10 monclient: dump: 2024-03-13T07:34:56.264 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.264 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.264 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.264 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.264 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.264 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.264 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.265 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.265 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.265 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f894b7fe640 4 set_mon_vals no callback set 2024-03-13T07:34:56.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f895b45d640 5 monclient: authenticate success, global_id 4316 2024-03-13T07:34:56.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f895b45d640 10 monclient: _renew_subs 2024-03-13T07:34:56.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f895b45d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:56.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f895b45d640 1 -- 172.21.15.132:0/1812550876 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8954143590 con 0x7f89540b0220 2024-03-13T07:34:56.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f895b45d640 10 client.4316.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:56.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f895b45d640 10 monclient: _renew_subs 2024-03-13T07:34:56.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f895b45d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:56.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f895b45d640 1 -- 172.21.15.132:0/1812550876 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8954142de0 con 0x7f89540b0220 2024-03-13T07:34:56.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.261+0000 7f895b45d640 1 librados: init done 2024-03-13T07:34:56.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.262+0000 7f895b45d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:56.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.262+0000 7f895b45d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:56.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.262+0000 7f895b45d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:56.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.262+0000 7f895b45d640 1 -- 172.21.15.132:0/1812550876 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8954143790 con 0x7f89540b0220 2024-03-13T07:34:56.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.286+0000 7f8949ffb640 1 -- 172.21.15.132:0/1812550876 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f895001a070 con 0x7f89540b0220 2024-03-13T07:34:56.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.286+0000 7f8949ffb640 1 --2- 172.21.15.132:0/1812550876 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f893806d560 0x7f893806fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.286+0000 7f89589f8640 1 --2- 172.21.15.132:0/1812550876 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f893806d560 0x7f893806fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.286+0000 7f89589f8640 10 monclient: get_auth_request con 0x7f893806d560 auth_method 0 2024-03-13T07:34:56.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.287+0000 7f8949ffb640 10 client.4316.objecter ms_handle_connect 0x7f893806d560 2024-03-13T07:34:56.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.287+0000 7f89589f8640 1 --2- 172.21.15.132:0/1812550876 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f893806d560 0x7f893806fa20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8954138950 tx=0x7f8944009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:56.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.290+0000 7f8949ffb640 1 -- 172.21.15.132:0/1812550876 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8950056470 con 0x7f89540b0220 2024-03-13T07:34:56.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.290+0000 7f8949ffb640 10 client.4316.objecter ms_dispatch 0x7f8954000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:56.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.290+0000 7f8949ffb640 3 client.4316.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:56.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.290+0000 7f8949ffb640 3 client.4316.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:56.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.290+0000 7f8949ffb640 20 client.4316.objecter dump_active .. 0 homeless 2024-03-13T07:34:56.319 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-13T07:34:56.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.338+0000 7f87b2f1c640 10 monclient: get_monmap_and_config 2024-03-13T07:34:56.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.338+0000 7f87b2f1c640 10 monclient: build_initial_monmap 2024-03-13T07:34:56.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.338+0000 7f87b2f1c640 10 monclient: monmap: 2024-03-13T07:34:56.338 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:56.339 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.339 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:56.339407+0000 2024-03-13T07:34:56.339 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:56.339407+0000 2024-03-13T07:34:56.339 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:56.339 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.339 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:56.339 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:56.339 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:56.339 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:56.339 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:56.339 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.339+0000 7f87b2f1c640 1 Processor -- start 2024-03-13T07:34:56.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.339+0000 7f87b2f1c640 1 -- start start 2024-03-13T07:34:56.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.339+0000 7f87b2f1c640 10 monclient: init 2024-03-13T07:34:56.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.340+0000 7f87b2f1c640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:56.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.340+0000 7f87b2f1c640 10 monclient: _add_conns ranks=[2,4,1,0,3] 2024-03-13T07:34:56.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.340+0000 7f87b2f1c640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87ac0b5320 0x7f87ac0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.340+0000 7f87b2f1c640 10 monclient(hunting): picked mon.noname-b con 0x7f87ac0b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:56.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.340+0000 7f87b2f1c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f87ac0b1ac0 0x7f87ac0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.340+0000 7f87b2f1c640 10 monclient(hunting): picked mon.noname-c con 0x7f87ac0b1ac0 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:56.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.340+0000 7f87b2f1c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f87ac13f630 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:56.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.340+0000 7f87b2f1c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ac0b1150 con 0x7f87ac13f630 2024-03-13T07:34:56.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.340+0000 7f87b2f1c640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.340+0000 7f87b2f1c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ac0b5700 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.340+0000 7f87b2f1c640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.340+0000 7f87b2f1c640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac0b13c0 con 0x7f87ac0b5320 2024-03-13T07:34:56.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.340+0000 7f87b2f1c640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:56.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.340+0000 7f87b2f1c640 10 monclient(hunting): authenticate will time out at 1323.748411s 2024-03-13T07:34:56.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.341+0000 7f87abfff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f87ac0b1ac0 0x7f87ac0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.341+0000 7f87abfff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f87ac0b1ac0 0x7f87ac0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:56084/0 (socket says 172.21.15.132:56084) 2024-03-13T07:34:56.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.341+0000 7f87abfff640 1 -- 172.21.15.132:0/1403972411 learned_addr learned my addr 172.21.15.132:0/1403972411 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:56.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.341+0000 7f87abfff640 10 monclient(hunting): get_auth_request con 0x7f87ac0b1ac0 auth_method 0 2024-03-13T07:34:56.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.341+0000 7f87abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.341+0000 7f87abfff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.341+0000 7f87abfff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.342+0000 7f87abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.342+0000 7f87abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.342+0000 7f87abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.343+0000 7f87abfff640 10 monclient(hunting): handle_auth_done global_id 4333 payload 274 2024-03-13T07:34:56.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.343+0000 7f87abfff640 1 -- 172.21.15.132:0/1403972411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87ac0b5320 msgr2=0x7f87ac0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:56.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.343+0000 7f87abfff640 1 --2- 172.21.15.132:0/1403972411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87ac0b5320 0x7f87ac0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.343+0000 7f87abfff640 1 -- 172.21.15.132:0/1403972411 >> v1:172.21.15.195:6789/0 conn(0x7f87ac13f630 legacy=0x7f87ac13fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:56.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.343+0000 7f87abfff640 10 monclient: _finish_hunting 0 2024-03-13T07:34:56.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.343+0000 7f87abfff640 1 monclient: found mon.noname-c 2024-03-13T07:34:56.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.343+0000 7f87abfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:56.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.343+0000 7f87abfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T07:34:56.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.343+0000 7f87abfff640 1 -- 172.21.15.132:0/1403972411 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac0b16a0 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.346+0000 7f87abfff640 1 --2- 172.21.15.132:0/1403972411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87ac0b1ac0 0x7f87ac0b1f00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8798009870 tx=0x7f879802b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e88fd9a155f60c39 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:56.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87ab7fe640 1 -- 172.21.15.132:0/1403972411 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798038070 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87ab7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87ab7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:56.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87ab7fe640 10 monclient: dump: 2024-03-13T07:34:56.350 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.351 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.351 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.351 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.351 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.351 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.351 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.351 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.351 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.351 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87ab7fe640 10 monclient: _finish_auth 0 2024-03-13T07:34:56.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87ab7fe640 10 monclient: _check_auth_tickets 2024-03-13T07:34:56.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87ab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:56.352 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87ab7fe640 1 -- 172.21.15.132:0/1403972411 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8798031740 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.352 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87b2f1c640 5 monclient: authenticate success, global_id 4333 2024-03-13T07:34:56.352 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87b2f1c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:56.352 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87ab7fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:56.352 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87b2f1c640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:56.352 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87ab7fe640 1 -- 172.21.15.132:0/1403972411 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879802f050 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.352 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87ab7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.352 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87b2f1c640 4 set_mon_vals no callback set 2024-03-13T07:34:56.352 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87ab7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:56.352 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.350+0000 7f87ab7fe640 10 monclient: dump: 2024-03-13T07:34:56.352 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.352 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.352 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.352 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.352 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.353 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.353 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.353 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.353 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.353 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.351+0000 7f87b2f1c640 10 monclient: shutdown 2024-03-13T07:34:56.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.351+0000 7f87b2f1c640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:56.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.351+0000 7f87b2f1c640 1 -- 172.21.15.132:0/1403972411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87ac0b1ac0 msgr2=0x7f87ac0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.351+0000 7f87b2f1c640 1 --2- 172.21.15.132:0/1403972411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87ac0b1ac0 0x7f87ac0b1f00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8798009870 tx=0x7f879802b7e0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.351+0000 7f87b2f1c640 1 -- 172.21.15.132:0/1403972411 shutdown_connections 2024-03-13T07:34:56.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.351+0000 7f87b2f1c640 1 --2- 172.21.15.132:0/1403972411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87ac0b1ac0 0x7f87ac0b1f00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.351+0000 7f87b2f1c640 1 --2- 172.21.15.132:0/1403972411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87ac0b5320 0x7f87ac0b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.351+0000 7f87b2f1c640 1 -- 172.21.15.132:0/1403972411 >> 172.21.15.132:0/1403972411 conn(0x7f87ac06d2b0 msgr2=0x7f87ac06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:56.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.351+0000 7f87b2f1c640 1 -- 172.21.15.132:0/1403972411 shutdown_connections 2024-03-13T07:34:56.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.351+0000 7f87b2f1c640 1 -- 172.21.15.132:0/1403972411 wait complete. 2024-03-13T07:34:56.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.352+0000 7f87b2f1c640 10 monclient: build_initial_monmap 2024-03-13T07:34:56.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.352+0000 7f87b2f1c640 10 monclient: monmap: 2024-03-13T07:34:56.357 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:56.357 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:56.357 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:56.357 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:56.357 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:56.357 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.357 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:56.357 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:56.357 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:56.357 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.353+0000 7f87b2f1c640 1 librados: starting msgr at 2024-03-13T07:34:56.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.353+0000 7f87b2f1c640 1 librados: starting objecter 2024-03-13T07:34:56.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.353+0000 7f87b2f1c640 1 Processor -- start 2024-03-13T07:34:56.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.353+0000 7f87b2f1c640 1 -- start start 2024-03-13T07:34:56.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.353+0000 7f87b2f1c640 1 librados: setting wanted keys 2024-03-13T07:34:56.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.353+0000 7f87b2f1c640 1 librados: calling monclient init 2024-03-13T07:34:56.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.353+0000 7f87b2f1c640 10 monclient: init 2024-03-13T07:34:56.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.354+0000 7f87b2f1c640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:56.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.354+0000 7f87b2f1c640 10 monclient: _add_conns ranks=[1,2,0] 2024-03-13T07:34:56.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.354+0000 7f87b2f1c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87ac0b1ac0 0x7f87ac13b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.354+0000 7f87b2f1c640 10 monclient(hunting): picked mon.noname-b con 0x7f87ac0b1ac0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:56.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.354+0000 7f87b2f1c640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87ac0b5320 0x7f87ac1408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.354+0000 7f87b2f1c640 10 monclient(hunting): picked mon.noname-c con 0x7f87ac0b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:56.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.354+0000 7f87b2f1c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87ac13f630 0x7f87ac1351d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.354+0000 7f87b2f1c640 10 monclient(hunting): picked mon.noname-a con 0x7f87ac13f630 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:56.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.354+0000 7f87b2f1c640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.354+0000 7f87b2f1c640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac0b4980 con 0x7f87ac13f630 2024-03-13T07:34:56.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.354+0000 7f87b2f1c640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.354+0000 7f87b2f1c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac0b4800 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.354+0000 7f87b2f1c640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.354+0000 7f87b2f1c640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac14cef0 con 0x7f87ac0b5320 2024-03-13T07:34:56.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.354+0000 7f87b2f1c640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:56.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.354+0000 7f87b2f1c640 10 monclient(hunting): authenticate will time out at 1323.762422s 2024-03-13T07:34:56.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.355+0000 7f87b0cb8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87ac0b1ac0 0x7f87ac13b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.355+0000 7f87b14b9640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87ac13f630 0x7f87ac1351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.355+0000 7f87b0cb8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87ac0b1ac0 0x7f87ac13b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:56096/0 (socket says 172.21.15.132:56096) 2024-03-13T07:34:56.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.355+0000 7f87b0cb8640 1 -- 172.21.15.132:0/1406460207 learned_addr learned my addr 172.21.15.132:0/1406460207 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:56.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.355+0000 7f87b14b9640 10 monclient(hunting): get_auth_request con 0x7f87ac13f630 auth_method 0 2024-03-13T07:34:56.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.355+0000 7f87b14b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.355+0000 7f87b14b9640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.355+0000 7f87b14b9640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.355+0000 7f87b0cb8640 10 monclient(hunting): get_auth_request con 0x7f87ac0b1ac0 auth_method 0 2024-03-13T07:34:56.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.355+0000 7f87b0cb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.355+0000 7f87b0cb8640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.355+0000 7f87b0cb8640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.355+0000 7f87b0cb8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.355+0000 7f87b0cb8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.355+0000 7f87b0cb8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.355+0000 7f87b14b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.356+0000 7f87b14b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.356+0000 7f87b14b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.358+0000 7f87b0cb8640 10 monclient(hunting): handle_auth_done global_id 4336 payload 824 2024-03-13T07:34:56.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.358+0000 7f87b0cb8640 1 -- 172.21.15.132:0/1406460207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87ac0b5320 msgr2=0x7f87ac1408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.358+0000 7f87b0cb8640 1 --2- 172.21.15.132:0/1406460207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87ac0b5320 0x7f87ac1408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.358+0000 7f87b0cb8640 1 -- 172.21.15.132:0/1406460207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87ac13f630 msgr2=0x7f87ac1351d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.358+0000 7f87b0cb8640 1 --2- 172.21.15.132:0/1406460207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87ac13f630 0x7f87ac1351d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.358+0000 7f87b0cb8640 10 monclient: _finish_hunting 0 2024-03-13T07:34:56.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.358+0000 7f87b0cb8640 1 monclient: found mon.noname-b 2024-03-13T07:34:56.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.358+0000 7f87b0cb8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:56.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.358+0000 7f87b0cb8640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:56.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.358+0000 7f87b0cb8640 1 -- 172.21.15.132:0/1406460207 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac0aa1a0 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.360+0000 7f87b0cb8640 1 --2- 172.21.15.132:0/1406460207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87ac0b1ac0 0x7f87ac13b7c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87a000b4a0 tx=0x7f87a000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:56.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.360+0000 7f87a97fa640 10 client.?.objecter ms_handle_connect 0x7f87ac0b1ac0 2024-03-13T07:34:56.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.360+0000 7f87a97fa640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87a97fa640 1 -- 172.21.15.132:0/1406460207 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0014070 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87a97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87a97fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87a97fa640 10 monclient: dump: 2024-03-13T07:34:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.364 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.365 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.365 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.365 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.365 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87a97fa640 10 monclient: _finish_auth 0 2024-03-13T07:34:56.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87a97fa640 10 monclient: _check_auth_tickets 2024-03-13T07:34:56.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87a97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:56.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87a97fa640 1 -- 172.21.15.132:0/1406460207 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a0010610 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87a97fa640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:56.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87a97fa640 1 -- 172.21.15.132:0/1406460207 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0010a00 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87b2f1c640 5 monclient: authenticate success, global_id 4336 2024-03-13T07:34:56.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87b2f1c640 10 monclient: _renew_subs 2024-03-13T07:34:56.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87b2f1c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:56.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87b2f1c640 1 -- 172.21.15.132:0/1406460207 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ac141320 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87aaffd640 4 set_mon_vals no callback set 2024-03-13T07:34:56.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87a97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87a97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:56.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87a97fa640 10 monclient: dump: 2024-03-13T07:34:56.366 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.366 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.366 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.366 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.366 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.366 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.366 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.366 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.366 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.366 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87b2f1c640 10 client.4336.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:56.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87b2f1c640 10 monclient: _renew_subs 2024-03-13T07:34:56.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87b2f1c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:56.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87b2f1c640 1 -- 172.21.15.132:0/1406460207 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ac135710 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.364+0000 7f87b2f1c640 1 librados: init done 2024-03-13T07:34:56.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.365+0000 7f87b2f1c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.365+0000 7f87b2f1c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.365+0000 7f87b2f1c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:56.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.365+0000 7f87b2f1c640 1 -- 172.21.15.132:0/1406460207 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ac04f000 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.424+0000 7fdcda245640 10 monclient: get_monmap_and_config 2024-03-13T07:34:56.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.424+0000 7fdcda245640 10 monclient: build_initial_monmap 2024-03-13T07:34:56.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.424+0000 7fdcda245640 10 monclient: monmap: 2024-03-13T07:34:56.424 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:56.425 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.425 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:56.425278+0000 2024-03-13T07:34:56.425 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:56.425278+0000 2024-03-13T07:34:56.425 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:56.425 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.425 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:56.425 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:56.425 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:56.425 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:56.425 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:56.425 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 1 Processor -- start 2024-03-13T07:34:56.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 1 -- start start 2024-03-13T07:34:56.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 10 monclient: init 2024-03-13T07:34:56.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:56.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 10 monclient: _add_conns ranks=[4,3,2,0,1] 2024-03-13T07:34:56.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdcd40b5320 0x7fdcd40b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 10 monclient(hunting): picked mon.noname-c con 0x7fdcd40b5320 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:56.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdcd40b1ac0 0x7fdcd40b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 10 monclient(hunting): picked mon.noname-a con 0x7fdcd40b1ac0 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:56.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdcd413f630 0x7fdcd413fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 10 monclient(hunting): picked mon.noname-b con 0x7fdcd413f630 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:56.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd40b1150 con 0x7fdcd40b1ac0 2024-03-13T07:34:56.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd413ffa0 con 0x7fdcd40b5320 2024-03-13T07:34:56.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40b13c0 con 0x7fdcd413f630 2024-03-13T07:34:56.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:56.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.425+0000 7fdcda245640 10 monclient(hunting): authenticate will time out at 1323.834047s 2024-03-13T07:34:56.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.426+0000 7fdcd2ffd640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdcd40b1ac0 0x7fdcd40b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.426+0000 7fdcd2ffd640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdcd40b1ac0 0x7fdcd40b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34576/0 (socket says 172.21.15.132:34576) 2024-03-13T07:34:56.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.426+0000 7fdcd2ffd640 1 -- 172.21.15.132:0/2413156260 learned_addr learned my addr 172.21.15.132:0/2413156260 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:56.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.426+0000 7fdcd2ffd640 10 monclient(hunting): get_auth_request con 0x7fdcd40b1ac0 auth_method 0 2024-03-13T07:34:56.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.426+0000 7fdcd2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.426+0000 7fdcd2ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.426+0000 7fdcd2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.427+0000 7fdcd2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.427+0000 7fdcd2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.427+0000 7fdcd2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.429+0000 7fdcd2ffd640 10 monclient(hunting): handle_auth_done global_id 4332 payload 274 2024-03-13T07:34:56.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.429+0000 7fdcd2ffd640 1 -- 172.21.15.132:0/2413156260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdcd413f630 msgr2=0x7fdcd413fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:56.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.429+0000 7fdcd2ffd640 1 --2- 172.21.15.132:0/2413156260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdcd413f630 0x7fdcd413fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.429+0000 7fdcd2ffd640 1 -- 172.21.15.132:0/2413156260 >> v2:172.21.15.195:3300/0 conn(0x7fdcd40b5320 msgr2=0x7fdcd40b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:56.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.429+0000 7fdcd2ffd640 1 --2- 172.21.15.132:0/2413156260 >> v2:172.21.15.195:3300/0 conn(0x7fdcd40b5320 0x7fdcd40b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.429+0000 7fdcd2ffd640 10 monclient: _finish_hunting 0 2024-03-13T07:34:56.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.429+0000 7fdcd2ffd640 1 monclient: found mon.noname-a 2024-03-13T07:34:56.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.429+0000 7fdcd2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:56.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.429+0000 7fdcd2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:56.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.429+0000 7fdcd2ffd640 1 -- 172.21.15.132:0/2413156260 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd40b16a0 con 0x7fdcd40b1ac0 2024-03-13T07:34:56.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.431+0000 7fdcd2ffd640 1 --2- 172.21.15.132:0/2413156260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd40b1ac0 0x7fdcd40b1f30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40b90f0 tx=0x7fdcc802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743cc8b9b44af33e server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:56.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcd27fc640 1 -- 172.21.15.132:0/2413156260 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8038070 con 0x7fdcd40b1ac0 2024-03-13T07:34:56.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcd27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcd27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:56.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcd27fc640 10 monclient: dump: 2024-03-13T07:34:56.435 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.435 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.435 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.435 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.435 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.435 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.435 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.435 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.435 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.435 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcd27fc640 10 monclient: _finish_auth 0 2024-03-13T07:34:56.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcd27fc640 10 monclient: _check_auth_tickets 2024-03-13T07:34:56.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcd27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:56.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcd27fc640 1 -- 172.21.15.132:0/2413156260 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc802f050 con 0x7fdcd40b1ac0 2024-03-13T07:34:56.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcda245640 5 monclient: authenticate success, global_id 4332 2024-03-13T07:34:56.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcda245640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:56.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcd27fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:56.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcda245640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:56.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcda245640 4 set_mon_vals no callback set 2024-03-13T07:34:56.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcd27fc640 1 -- 172.21.15.132:0/2413156260 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8031970 con 0x7fdcd40b1ac0 2024-03-13T07:34:56.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcd27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcd27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:56.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.434+0000 7fdcd27fc640 10 monclient: dump: 2024-03-13T07:34:56.436 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.436 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.436 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.437 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.437 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.437 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.437 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.437 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.437 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.437 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.439 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.435+0000 7fdcda245640 10 monclient: shutdown 2024-03-13T07:34:56.439 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.435+0000 7fdcda245640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:56.439 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.435+0000 7fdcda245640 1 -- 172.21.15.132:0/2413156260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd40b1ac0 msgr2=0x7fdcd40b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.439 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.435+0000 7fdcda245640 1 --2- 172.21.15.132:0/2413156260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd40b1ac0 0x7fdcd40b1f30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40b90f0 tx=0x7fdcc802b7f0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.439 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.435+0000 7fdcda245640 1 -- 172.21.15.132:0/2413156260 shutdown_connections 2024-03-13T07:34:56.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.435+0000 7fdcda245640 1 --2- 172.21.15.132:0/2413156260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdcd413f630 0x7fdcd413fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.435+0000 7fdcda245640 1 --2- 172.21.15.132:0/2413156260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd40b1ac0 0x7fdcd40b1f30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.435+0000 7fdcda245640 1 --2- 172.21.15.132:0/2413156260 >> v2:172.21.15.195:3300/0 conn(0x7fdcd40b5320 0x7fdcd40b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.435+0000 7fdcda245640 1 -- 172.21.15.132:0/2413156260 >> 172.21.15.132:0/2413156260 conn(0x7fdcd406d2b0 msgr2=0x7fdcd406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:56.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.435+0000 7fdcda245640 1 -- 172.21.15.132:0/2413156260 shutdown_connections 2024-03-13T07:34:56.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.435+0000 7fdcda245640 1 -- 172.21.15.132:0/2413156260 wait complete. 2024-03-13T07:34:56.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.436+0000 7fdcda245640 10 monclient: build_initial_monmap 2024-03-13T07:34:56.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.436+0000 7fdcda245640 10 monclient: monmap: 2024-03-13T07:34:56.440 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:56.440 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:56.440 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:56.440 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:56.440 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:56.440 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.440 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:56.441 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:56.441 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:56.441 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.441 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.436+0000 7fdcda245640 1 librados: starting msgr at 2024-03-13T07:34:56.441 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.436+0000 7fdcda245640 1 librados: starting objecter 2024-03-13T07:34:56.441 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 1 Processor -- start 2024-03-13T07:34:56.441 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 1 -- start start 2024-03-13T07:34:56.441 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 1 librados: setting wanted keys 2024-03-13T07:34:56.441 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 1 librados: calling monclient init 2024-03-13T07:34:56.441 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 10 monclient: init 2024-03-13T07:34:56.441 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:56.441 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 10 monclient: _add_conns ranks=[2,1,0] 2024-03-13T07:34:56.441 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdcd40b1ac0 0x7fdcd40aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.441 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 10 monclient(hunting): picked mon.noname-c con 0x7fdcd40b1ac0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:56.441 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcd40b5320 0x7fdcd40ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 10 monclient(hunting): picked mon.noname-b con 0x7fdcd40b5320 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:56.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd413f630 0x7fdcd41dafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 10 monclient(hunting): picked mon.noname-a con 0x7fdcd413f630 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:56.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40b4800 con 0x7fdcd413f630 2024-03-13T07:34:56.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40afd00 con 0x7fdcd40b5320 2024-03-13T07:34:56.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40b4980 con 0x7fdcd40b1ac0 2024-03-13T07:34:56.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:56.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.437+0000 7fdcda245640 10 monclient(hunting): authenticate will time out at 1323.845868s 2024-03-13T07:34:56.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.438+0000 7fdcd3fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd413f630 0x7fdcd41dafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.438+0000 7fdcd3fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd413f630 0x7fdcd41dafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34588/0 (socket says 172.21.15.132:34588) 2024-03-13T07:34:56.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.438+0000 7fdcd3fff640 1 -- 172.21.15.132:0/2313796398 learned_addr learned my addr 172.21.15.132:0/2313796398 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:56.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.438+0000 7fdcd3fff640 10 monclient(hunting): get_auth_request con 0x7fdcd413f630 auth_method 0 2024-03-13T07:34:56.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.438+0000 7fdcd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.438+0000 7fdcd3fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.438+0000 7fdcd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.439+0000 7fdcd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.439+0000 7fdcd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.439+0000 7fdcd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.441+0000 7fdcd3fff640 10 monclient(hunting): handle_auth_done global_id 4335 payload 824 2024-03-13T07:34:56.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.441+0000 7fdcd3fff640 1 -- 172.21.15.132:0/2313796398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdcd40b1ac0 msgr2=0x7fdcd40aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.441+0000 7fdcd3fff640 1 --2- 172.21.15.132:0/2313796398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdcd40b1ac0 0x7fdcd40aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.441+0000 7fdcd3fff640 1 -- 172.21.15.132:0/2313796398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcd40b5320 msgr2=0x7fdcd40ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.441+0000 7fdcd3fff640 1 --2- 172.21.15.132:0/2313796398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcd40b5320 0x7fdcd40ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.441+0000 7fdcd3fff640 10 monclient: _finish_hunting 0 2024-03-13T07:34:56.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.441+0000 7fdcd3fff640 1 monclient: found mon.noname-a 2024-03-13T07:34:56.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.441+0000 7fdcd3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:56.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.441+0000 7fdcd3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:56.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.441+0000 7fdcd3fff640 1 -- 172.21.15.132:0/2313796398 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd40ac690 con 0x7fdcd413f630 2024-03-13T07:34:56.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.444+0000 7fdcd3fff640 1 --2- 172.21.15.132:0/2313796398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd413f630 0x7fdcd41dafe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00b970 tx=0x7fdccc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:56.444 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.444+0000 7fdcbbfff640 10 client.?.objecter ms_handle_connect 0x7fdcd413f630 2024-03-13T07:34:56.445 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.444+0000 7fdcbbfff640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:56.446 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.446+0000 7fdcbbfff640 1 -- 172.21.15.132:0/2313796398 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc019070 con 0x7fdcd413f630 2024-03-13T07:34:56.446 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.446+0000 7fdcbbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.446+0000 7fdcbbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.446+0000 7fdcbbfff640 10 monclient: dump: 2024-03-13T07:34:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.447 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.446+0000 7fdcbbfff640 10 monclient: _finish_auth 0 2024-03-13T07:34:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.446+0000 7fdcbbfff640 10 monclient: _check_auth_tickets 2024-03-13T07:34:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.446+0000 7fdcbbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:56.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.446+0000 7fdcda245640 5 monclient: authenticate success, global_id 4335 2024-03-13T07:34:56.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.447+0000 7fdcda245640 10 monclient: _renew_subs 2024-03-13T07:34:56.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.447+0000 7fdcda245640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:56.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.447+0000 7fdcda245640 1 -- 172.21.15.132:0/2313796398 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd40aacd0 con 0x7fdcd413f630 2024-03-13T07:34:56.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.447+0000 7fdcda245640 10 client.4335.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:56.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.447+0000 7fdcda245640 10 monclient: _renew_subs 2024-03-13T07:34:56.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.447+0000 7fdcda245640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:56.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.447+0000 7fdcda245640 1 -- 172.21.15.132:0/2313796398 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd41db520 con 0x7fdcd413f630 2024-03-13T07:34:56.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.447+0000 7fdcda245640 1 librados: init done 2024-03-13T07:34:56.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.447+0000 7fdcda245640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:56.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.447+0000 7fdcda245640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:56.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.447+0000 7fdcda245640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:56.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.447+0000 7fdcda245640 1 -- 172.21.15.132:0/2313796398 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd4146720 con 0x7fdcd413f630 2024-03-13T07:34:56.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.448+0000 7fdcbbfff640 1 -- 172.21.15.132:0/2313796398 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdccc00cb40 con 0x7fdcd413f630 2024-03-13T07:34:56.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.448+0000 7fdcbbfff640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:56.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.448+0000 7fdcbbfff640 1 -- 172.21.15.132:0/2313796398 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc023060 con 0x7fdcd413f630 2024-03-13T07:34:56.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.448+0000 7fdcbbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.448+0000 7fdcbbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:56.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.448+0000 7fdcbbfff640 10 monclient: dump: 2024-03-13T07:34:56.450 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.450 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.450 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.450 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.450 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.450 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.450 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.450 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.450 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.450 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.448+0000 7fdcd1ffb640 4 set_mon_vals no callback set 2024-03-13T07:34:56.505 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-13T07:34:56.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.575+0000 7f8949ffb640 1 -- 172.21.15.132:0/1812550876 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f8950021080 con 0x7f89540b0220 2024-03-13T07:34:56.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.575+0000 7f8949ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:56.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.575+0000 7f8949ffb640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:56.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.612+0000 7f87a97fa640 1 -- 172.21.15.132:0/1406460207 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f87a0017070 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.612+0000 7f87a97fa640 1 --2- 172.21.15.132:0/1406460207 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f879006d5b0 0x7f879006fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.612+0000 7f87abfff640 1 --2- 172.21.15.132:0/1406460207 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f879006d5b0 0x7f879006fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.612+0000 7f87abfff640 10 monclient: get_auth_request con 0x7f879006d5b0 auth_method 0 2024-03-13T07:34:56.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.613+0000 7f87a97fa640 10 client.4336.objecter ms_handle_connect 0x7f879006d5b0 2024-03-13T07:34:56.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.613+0000 7f87abfff640 1 --2- 172.21.15.132:0/1406460207 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f879006d5b0 0x7f879006fa70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8798035000 tx=0x7f8798030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:56.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.614+0000 7f87a97fa640 1 -- 172.21.15.132:0/1406460207 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f87a0056d60 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.614+0000 7f87a97fa640 10 client.4336.objecter ms_dispatch 0x7f87ac000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:56.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.614+0000 7f87a97fa640 3 client.4336.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:56.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.614+0000 7f87a97fa640 3 client.4336.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:56.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.615+0000 7f87a97fa640 20 client.4336.objecter dump_active .. 0 homeless 2024-03-13T07:34:56.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.615+0000 7f19f3c47640 10 monclient: get_monmap_and_config 2024-03-13T07:34:56.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.615+0000 7f19f3c47640 10 monclient: build_initial_monmap 2024-03-13T07:34:56.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.615+0000 7f19f3c47640 10 monclient: monmap: 2024-03-13T07:34:56.616 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:56.616 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.616 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:56.616159+0000 2024-03-13T07:34:56.616 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:56.616159+0000 2024-03-13T07:34:56.616 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:56.616 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.616 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:56.616 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:56.616 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:56.616 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:56.616 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:56.616 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.615+0000 7f19f3c47640 1 Processor -- start 2024-03-13T07:34:56.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.616+0000 7f19f3c47640 1 -- start start 2024-03-13T07:34:56.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.616+0000 7f19f3c47640 10 monclient: init 2024-03-13T07:34:56.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.616+0000 7f19f3c47640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:56.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.616+0000 7f19f3c47640 10 monclient: _add_conns ranks=[2,1,0,3,4] 2024-03-13T07:34:56.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.616+0000 7f19f3c47640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19ec0b5320 0x7f19ec0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.616+0000 7f19f3c47640 10 monclient(hunting): picked mon.noname-b con 0x7f19ec0b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:56.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.616+0000 7f19f3c47640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f19ec0b1ac0 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:56.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.616+0000 7f19f3c47640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f19ec13fd40 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:56.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.616+0000 7f19f3c47640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec0b1150 con 0x7f19ec13fd40 2024-03-13T07:34:56.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.616+0000 7f19f3c47640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec0b13c0 con 0x7f19ec0b1ac0 2024-03-13T07:34:56.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.616+0000 7f19f3c47640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.616+0000 7f19f3c47640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0b16a0 con 0x7f19ec0b5320 2024-03-13T07:34:56.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.616+0000 7f19f3c47640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:56.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.616+0000 7f19f3c47640 10 monclient(hunting): authenticate will time out at 1324.024924s 2024-03-13T07:34:56.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.617+0000 7f19f19e3640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19ec0b5320 0x7f19ec0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.617+0000 7f19f19e3640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19ec0b5320 0x7f19ec0b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:57818/0 (socket says 172.21.15.132:57818) 2024-03-13T07:34:56.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.617+0000 7f19f19e3640 1 -- 172.21.15.132:0/3924896650 learned_addr learned my addr 172.21.15.132:0/3924896650 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:56.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.617+0000 7f19f19e3640 10 monclient(hunting): get_auth_request con 0x7f19ec0b5320 auth_method 0 2024-03-13T07:34:56.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.617+0000 7f19f19e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.617+0000 7f19f19e3640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.617+0000 7f19f19e3640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.618+0000 7f19f19e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.618+0000 7f19f19e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.618+0000 7f19f19e3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.620+0000 7f19f19e3640 10 monclient(hunting): handle_auth_done global_id 4319 payload 274 2024-03-13T07:34:56.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.620+0000 7f19f19e3640 1 -- 172.21.15.132:0/3924896650 >> v1:172.21.15.195:6789/0 conn(0x7f19ec0b1ac0 legacy=0x7f19ec13f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:56.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.620+0000 7f19f19e3640 1 -- 172.21.15.132:0/3924896650 >> v1:172.21.15.132:6789/0 conn(0x7f19ec13fd40 legacy=0x7f19ec14cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:56.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.620+0000 7f19f19e3640 10 monclient: _finish_hunting 0 2024-03-13T07:34:56.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.620+0000 7f19f19e3640 1 monclient: found mon.noname-b 2024-03-13T07:34:56.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.620+0000 7f19f19e3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:56.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.620+0000 7f19f19e3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T07:34:56.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.620+0000 7f19f19e3640 1 -- 172.21.15.132:0/3924896650 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec0afa20 con 0x7f19ec0b5320 2024-03-13T07:34:56.621 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.622+0000 7f19f19e3640 1 --2- 172.21.15.132:0/3924896650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19ec0b5320 0x7f19ec0b0220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f19e80098a0 tx=0x7f19e802b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2dc77e1a516640d server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:56.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.624+0000 7f87a97fa640 1 -- 172.21.15.132:0/1406460207 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f87a0058cc0 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.624+0000 7f87a97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:56.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.624+0000 7f87a97fa640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:56.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.626+0000 7f19f09e1640 1 -- 172.21.15.132:0/3924896650 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8038070 con 0x7f19ec0b5320 2024-03-13T07:34:56.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.626+0000 7f19f09e1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.626+0000 7f19f09e1640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:56.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.626+0000 7f19f09e1640 10 monclient: dump: 2024-03-13T07:34:56.627 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.627 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.627 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.627 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.627 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.627 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.627 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.627 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.627 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.627 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.626+0000 7f19f09e1640 10 monclient: _finish_auth 0 2024-03-13T07:34:56.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.626+0000 7f19f09e1640 10 monclient: _check_auth_tickets 2024-03-13T07:34:56.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.626+0000 7f19f09e1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:56.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.626+0000 7f19f09e1640 1 -- 172.21.15.132:0/3924896650 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e8031520 con 0x7f19ec0b5320 2024-03-13T07:34:56.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.626+0000 7f19f09e1640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:56.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.626+0000 7f19f09e1640 1 -- 172.21.15.132:0/3924896650 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e80318f0 con 0x7f19ec0b5320 2024-03-13T07:34:56.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.626+0000 7f19f09e1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.626+0000 7f19f09e1640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:56.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.626+0000 7f19f09e1640 10 monclient: dump: 2024-03-13T07:34:56.628 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.628 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.628 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.628 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.628 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.628 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.628 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.628 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.629 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.629 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.626+0000 7f19f3c47640 5 monclient: authenticate success, global_id 4319 2024-03-13T07:34:56.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.626+0000 7f19f3c47640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:56.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.626+0000 7f19f3c47640 4 set_mon_vals no callback set 2024-03-13T07:34:56.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.627+0000 7f19f3c47640 10 monclient: shutdown 2024-03-13T07:34:56.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.627+0000 7f19f3c47640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:56.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.627+0000 7f19f3c47640 1 -- 172.21.15.132:0/3924896650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19ec0b5320 msgr2=0x7f19ec0b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.627+0000 7f19f3c47640 1 --2- 172.21.15.132:0/3924896650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19ec0b5320 0x7f19ec0b0220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f19e80098a0 tx=0x7f19e802b7c0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.627+0000 7f19f3c47640 1 -- 172.21.15.132:0/3924896650 shutdown_connections 2024-03-13T07:34:56.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.627+0000 7f19f3c47640 1 --2- 172.21.15.132:0/3924896650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19ec0b5320 0x7f19ec0b0220 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.627+0000 7f19f3c47640 1 -- 172.21.15.132:0/3924896650 >> 172.21.15.132:0/3924896650 conn(0x7f19ec06d2b0 msgr2=0x7f19ec06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:56.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.627+0000 7f19f3c47640 1 -- 172.21.15.132:0/3924896650 shutdown_connections 2024-03-13T07:34:56.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.627+0000 7f19f3c47640 1 -- 172.21.15.132:0/3924896650 wait complete. 2024-03-13T07:34:56.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.628+0000 7f19f3c47640 10 monclient: build_initial_monmap 2024-03-13T07:34:56.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.628+0000 7f19f3c47640 10 monclient: monmap: 2024-03-13T07:34:56.632 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:56.632 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:56.632 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:56.632 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:56.632 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:56.632 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.632 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:56.632 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:56.632 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:56.632 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.628+0000 7f19f3c47640 1 librados: starting msgr at 2024-03-13T07:34:56.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.628+0000 7f19f3c47640 1 librados: starting objecter 2024-03-13T07:34:56.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.628+0000 7f19f3c47640 1 Processor -- start 2024-03-13T07:34:56.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.629+0000 7f19f3c47640 1 -- start start 2024-03-13T07:34:56.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.629+0000 7f19f3c47640 1 librados: setting wanted keys 2024-03-13T07:34:56.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.629+0000 7f19f3c47640 1 librados: calling monclient init 2024-03-13T07:34:56.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.629+0000 7f19f3c47640 10 monclient: init 2024-03-13T07:34:56.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f3c47640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:56.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f3c47640 10 monclient: _add_conns ranks=[1,2,0] 2024-03-13T07:34:56.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f3c47640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19ec0b1ac0 0x7f19ec146640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f3c47640 10 monclient(hunting): picked mon.noname-b con 0x7f19ec0b1ac0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:56.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f3c47640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19ec0b5320 0x7f19ec145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f3c47640 10 monclient(hunting): picked mon.noname-c con 0x7f19ec0b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:56.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f3c47640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19ec13fd40 0x7f19ec143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f3c47640 10 monclient(hunting): picked mon.noname-a con 0x7f19ec13fd40 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:56.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f3c47640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f3c47640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0b1f00 con 0x7f19ec13fd40 2024-03-13T07:34:56.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f3c47640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f3c47640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0b4840 con 0x7f19ec0b1ac0 2024-03-13T07:34:56.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f3c47640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f3c47640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0b2080 con 0x7f19ec0b5320 2024-03-13T07:34:56.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f3c47640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:56.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f3c47640 10 monclient(hunting): authenticate will time out at 1324.038814s 2024-03-13T07:34:56.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.630+0000 7f19f19e3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19ec0b1ac0 0x7f19ec146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.631+0000 7f19f19e3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19ec0b1ac0 0x7f19ec146640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:56112/0 (socket says 172.21.15.132:56112) 2024-03-13T07:34:56.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.631+0000 7f19f19e3640 1 -- 172.21.15.132:0/1537107393 learned_addr learned my addr 172.21.15.132:0/1537107393 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:56.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.631+0000 7f19f19e3640 10 monclient(hunting): get_auth_request con 0x7f19ec0b1ac0 auth_method 0 2024-03-13T07:34:56.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.631+0000 7f19f19e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.631+0000 7f19f19e3640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.631+0000 7f19f19e3640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.631+0000 7f19f11e2640 1 --2- 172.21.15.132:0/1537107393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19ec0b5320 0x7f19ec145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.631+0000 7f19f19e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.631+0000 7f19f19e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.631+0000 7f19f19e3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.631+0000 7f19f11e2640 10 monclient(hunting): get_auth_request con 0x7f19ec0b5320 auth_method 0 2024-03-13T07:34:56.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.631+0000 7f19f11e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.631+0000 7f19f11e2640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.631+0000 7f19f11e2640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.632+0000 7f19f11e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.632+0000 7f19f11e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.632+0000 7f19f11e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.634+0000 7f19f19e3640 10 monclient(hunting): handle_auth_done global_id 4339 payload 824 2024-03-13T07:34:56.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.634+0000 7f19f19e3640 1 -- 172.21.15.132:0/1537107393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19ec0b5320 msgr2=0x7f19ec145a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.634+0000 7f19f19e3640 1 --2- 172.21.15.132:0/1537107393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19ec0b5320 0x7f19ec145a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.634+0000 7f19f19e3640 1 -- 172.21.15.132:0/1537107393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19ec13fd40 msgr2=0x7f19ec143610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.634+0000 7f19f19e3640 1 --2- 172.21.15.132:0/1537107393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19ec13fd40 0x7f19ec143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.634+0000 7f19f19e3640 10 monclient: _finish_hunting 0 2024-03-13T07:34:56.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.634+0000 7f19f19e3640 1 monclient: found mon.noname-b 2024-03-13T07:34:56.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.634+0000 7f19f19e3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:56.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.634+0000 7f19f19e3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:56.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.634+0000 7f19f19e3640 1 -- 172.21.15.132:0/1537107393 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec06ea60 con 0x7f19ec0b1ac0 2024-03-13T07:34:56.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.637+0000 7f19f19e3640 1 --2- 172.21.15.132:0/1537107393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19ec0b1ac0 0x7f19ec146640 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f19e8000c00 tx=0x7f19e80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:56.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.637+0000 7f19e27fc640 10 client.?.objecter ms_handle_connect 0x7f19ec0b1ac0 2024-03-13T07:34:56.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.637+0000 7f19e27fc640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:56.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19e27fc640 1 -- 172.21.15.132:0/1537107393 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8043070 con 0x7f19ec0b1ac0 2024-03-13T07:34:56.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19e27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19e27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:56.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19e27fc640 10 monclient: dump: 2024-03-13T07:34:56.642 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.642 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.642 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.642 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.642 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.642 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.642 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.643 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.643 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.643 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19e27fc640 10 monclient: _finish_auth 0 2024-03-13T07:34:56.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19e27fc640 10 monclient: _check_auth_tickets 2024-03-13T07:34:56.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19e27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:56.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19e27fc640 1 -- 172.21.15.132:0/1537107393 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e80338a0 con 0x7f19ec0b1ac0 2024-03-13T07:34:56.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19e27fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:56.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19e27fc640 1 -- 172.21.15.132:0/1537107393 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8038040 con 0x7f19ec0b1ac0 2024-03-13T07:34:56.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19e27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19e27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:56.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19e27fc640 10 monclient: dump: 2024-03-13T07:34:56.643 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.643 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.643 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.644 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.644 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.644 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.644 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.644 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.644 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.644 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19f3c47640 5 monclient: authenticate success, global_id 4339 2024-03-13T07:34:56.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19e37fe640 4 set_mon_vals no callback set 2024-03-13T07:34:56.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19f3c47640 10 monclient: _renew_subs 2024-03-13T07:34:56.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19f3c47640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:56.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19f3c47640 1 -- 172.21.15.132:0/1537107393 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec06ed40 con 0x7f19ec0b1ac0 2024-03-13T07:34:56.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19f3c47640 10 client.4339.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:56.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19f3c47640 10 monclient: _renew_subs 2024-03-13T07:34:56.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19f3c47640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:56.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19f3c47640 1 -- 172.21.15.132:0/1537107393 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec143b50 con 0x7f19ec0b1ac0 2024-03-13T07:34:56.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.641+0000 7f19f3c47640 1 librados: init done 2024-03-13T07:34:56.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.642+0000 7f19f3c47640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:56.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.642+0000 7f19f3c47640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:56.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.642+0000 7f19f3c47640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:56.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.642+0000 7f19f3c47640 1 -- 172.21.15.132:0/1537107393 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec146170 con 0x7f19ec0b1ac0 2024-03-13T07:34:56.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.661+0000 7f19e27fc640 1 -- 172.21.15.132:0/1537107393 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f19e8046070 con 0x7f19ec0b1ac0 2024-03-13T07:34:56.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.661+0000 7f19e27fc640 1 --2- 172.21.15.132:0/1537107393 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f19d006d5b0 0x7f19d006fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.661+0000 7f19f11e2640 1 --2- 172.21.15.132:0/1537107393 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f19d006d5b0 0x7f19d006fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.661+0000 7f19f11e2640 10 monclient: get_auth_request con 0x7f19d006d5b0 auth_method 0 2024-03-13T07:34:56.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.662+0000 7f19e27fc640 10 client.4339.objecter ms_handle_connect 0x7f19d006d5b0 2024-03-13T07:34:56.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.661+0000 7f19f11e2640 1 --2- 172.21.15.132:0/1537107393 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f19d006d5b0 0x7f19d006fa70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0b3f40 tx=0x7f19dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:56.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.663+0000 7f19e27fc640 1 -- 172.21.15.132:0/1537107393 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f19e80790a0 con 0x7f19ec0b1ac0 2024-03-13T07:34:56.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.663+0000 7f19e27fc640 10 client.4339.objecter ms_dispatch 0x7f19ec000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:56.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.663+0000 7f19e27fc640 3 client.4339.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:56.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.663+0000 7f19e27fc640 3 client.4339.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:56.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.663+0000 7f19e27fc640 20 client.4339.objecter dump_active .. 0 homeless 2024-03-13T07:34:56.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.735+0000 7fdcbbfff640 1 -- 172.21.15.132:0/2313796398 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdccc01c070 con 0x7fdcd413f630 2024-03-13T07:34:56.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.735+0000 7fdcbbfff640 1 --2- 172.21.15.132:0/2313796398 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fdcc006d560 0x7fdcc006fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.735+0000 7fdcd37fe640 1 --2- 172.21.15.132:0/2313796398 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fdcc006d560 0x7fdcc006fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.735+0000 7fdcd37fe640 10 monclient: get_auth_request con 0x7fdcc006d560 auth_method 0 2024-03-13T07:34:56.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.736+0000 7fdcbbfff640 10 client.4335.objecter ms_handle_connect 0x7fdcc006d560 2024-03-13T07:34:56.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.736+0000 7fdcd37fe640 1 --2- 172.21.15.132:0/2313796398 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fdcc006d560 0x7fdcc006fa20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009710 tx=0x7fdcc4006c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:56.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.750+0000 7fdcbbfff640 1 -- 172.21.15.132:0/2313796398 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fdccc057490 con 0x7fdcd413f630 2024-03-13T07:34:56.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.750+0000 7fdcbbfff640 10 client.4335.objecter ms_dispatch 0x7fdcd4000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:56.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.750+0000 7fdcbbfff640 3 client.4335.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:56.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.750+0000 7fdcbbfff640 3 client.4335.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:56.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.750+0000 7fdcbbfff640 20 client.4335.objecter dump_active .. 0 homeless 2024-03-13T07:34:56.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.750+0000 7fdcbbfff640 1 -- 172.21.15.132:0/2313796398 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fdccc0593f0 con 0x7fdcd413f630 2024-03-13T07:34:56.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.750+0000 7fdcbbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:56.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.750+0000 7fdcbbfff640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:56.793 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-13T07:34:56.800 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-13T07:34:56.802 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-13T07:34:56.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.808+0000 7f895b45d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-03-13T07:34:56.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.809+0000 7f895b45d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-03-13T07:34:56.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.809+0000 7f895b45d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:56.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.809+0000 7f895b45d640 1 -- 172.21.15.132:0/1812550876 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f89541cc5a0 con 0x7f89540b0220 2024-03-13T07:34:56.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.816+0000 7f8949ffb640 1 -- 172.21.15.132:0/1812550876 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f89541cc5a0 con 0x7f89540b0220 2024-03-13T07:34:56.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.816+0000 7f8949ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-03-13T07:34:56.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.816+0000 7f8949ffb640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:56.816 INFO:teuthology.orchestra.run.smithi132.stdout:77309411338 2024-03-13T07:34:56.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 10 librados: watch_flush enter 2024-03-13T07:34:56.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 10 librados: watch_flush exit 2024-03-13T07:34:56.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 20 client.4316.objecter shutdown clearing up homeless session... 2024-03-13T07:34:56.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 10 client.4316.objecter successfully canceled tick 2024-03-13T07:34:56.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 1 -- 172.21.15.132:0/1812550876 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f893806d560 msgr2=0x7f893806fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 1 --2- 172.21.15.132:0/1812550876 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f893806d560 0x7f893806fa20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8954138950 tx=0x7f8944009240 comp rx=0 tx=0).stop 2024-03-13T07:34:56.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 10 monclient: shutdown 2024-03-13T07:34:56.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:56.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 1 -- 172.21.15.132:0/1812550876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89540b0220 msgr2=0x7f8954145d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 1 --2- 172.21.15.132:0/1812550876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89540b0220 0x7f8954145d40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f89540b5850 tx=0x7f895000c820 comp rx=0 tx=0).stop 2024-03-13T07:34:56.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 1 -- 172.21.15.132:0/1812550876 shutdown_connections 2024-03-13T07:34:56.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 1 --2- 172.21.15.132:0/1812550876 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f893806d560 0x7f893806fa20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 1 --2- 172.21.15.132:0/1812550876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f895413fb70 0x7f8954141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 1 --2- 172.21.15.132:0/1812550876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89540b5320 0x7f89541403e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 1 --2- 172.21.15.132:0/1812550876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89540b0220 0x7f8954145d40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 1 -- 172.21.15.132:0/1812550876 >> 172.21.15.132:0/1812550876 conn(0x7f895406d2b0 msgr2=0x7f89540b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:56.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 1 -- 172.21.15.132:0/1812550876 shutdown_connections 2024-03-13T07:34:56.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 1 -- 172.21.15.132:0/1812550876 wait complete. 2024-03-13T07:34:56.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.819+0000 7f895b45d640 1 librados: shutdown 2024-03-13T07:34:56.833 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411338 got 77309411338 for osd.4 2024-03-13T07:34:56.833 DEBUG:teuthology.parallel:result is None 2024-03-13T07:34:56.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.862+0000 7f87b2f1c640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-03-13T07:34:56.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.862+0000 7f87b2f1c640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-03-13T07:34:56.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.862+0000 7f87b2f1c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:56.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.862+0000 7f87b2f1c640 1 -- 172.21.15.132:0/1406460207 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f87ac0ae520 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.898+0000 7f19e27fc640 1 -- 172.21.15.132:0/1537107393 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f19e807b000 con 0x7f19ec0b1ac0 2024-03-13T07:34:56.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.898+0000 7f19e27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:56.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.898+0000 7f19e27fc640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:56.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.904+0000 7f87a97fa640 1 -- 172.21.15.132:0/1406460207 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f87a005cd60 con 0x7f87ac0b1ac0 2024-03-13T07:34:56.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.905+0000 7f87a97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-03-13T07:34:56.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.905+0000 7f87a97fa640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:56.905 INFO:teuthology.orchestra.run.smithi132.stdout:55834574855 2024-03-13T07:34:56.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.907+0000 7f87b2f1c640 10 librados: watch_flush enter 2024-03-13T07:34:56.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 10 librados: watch_flush exit 2024-03-13T07:34:56.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 20 client.4336.objecter shutdown clearing up homeless session... 2024-03-13T07:34:56.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 10 client.4336.objecter successfully canceled tick 2024-03-13T07:34:56.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 1 -- 172.21.15.132:0/1406460207 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f879006d5b0 msgr2=0x7f879006fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 1 --2- 172.21.15.132:0/1406460207 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f879006d5b0 0x7f879006fa70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8798035000 tx=0x7f8798030000 comp rx=0 tx=0).stop 2024-03-13T07:34:56.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 10 monclient: shutdown 2024-03-13T07:34:56.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:56.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 1 -- 172.21.15.132:0/1406460207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87ac0b1ac0 msgr2=0x7f87ac13b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 1 --2- 172.21.15.132:0/1406460207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87ac0b1ac0 0x7f87ac13b7c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87a000b4a0 tx=0x7f87a000b960 comp rx=0 tx=0).stop 2024-03-13T07:34:56.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 1 -- 172.21.15.132:0/1406460207 shutdown_connections 2024-03-13T07:34:56.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 1 --2- 172.21.15.132:0/1406460207 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f879006d5b0 0x7f879006fa70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 1 --2- 172.21.15.132:0/1406460207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87ac13f630 0x7f87ac1351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 1 --2- 172.21.15.132:0/1406460207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87ac0b5320 0x7f87ac1408a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 1 --2- 172.21.15.132:0/1406460207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87ac0b1ac0 0x7f87ac13b7c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 1 -- 172.21.15.132:0/1406460207 >> 172.21.15.132:0/1406460207 conn(0x7f87ac06d2b0 msgr2=0x7f87ac1388e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:56.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 1 -- 172.21.15.132:0/1406460207 shutdown_connections 2024-03-13T07:34:56.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 1 -- 172.21.15.132:0/1406460207 wait complete. 2024-03-13T07:34:56.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.908+0000 7f87b2f1c640 1 librados: shutdown 2024-03-13T07:34:56.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.917+0000 7fcb9a16a640 10 monclient: get_monmap_and_config 2024-03-13T07:34:56.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.917+0000 7fcb9a16a640 10 monclient: build_initial_monmap 2024-03-13T07:34:56.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.917+0000 7fcb9a16a640 10 monclient: monmap: 2024-03-13T07:34:56.917 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:56.917 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.917 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:56.918133+0000 2024-03-13T07:34:56.917 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:56.918133+0000 2024-03-13T07:34:56.917 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:56.917 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.917 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:56.917 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:56.917 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:56.918 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:56.918 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:56.918 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.918+0000 7fcb9a16a640 1 Processor -- start 2024-03-13T07:34:56.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.918+0000 7fcb9a16a640 1 -- start start 2024-03-13T07:34:56.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.918+0000 7fcb9a16a640 10 monclient: init 2024-03-13T07:34:56.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.919+0000 7fcb9a16a640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:56.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.919+0000 7fcb9a16a640 10 monclient: _add_conns ranks=[2,4,3,0,1] 2024-03-13T07:34:56.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.919+0000 7fcb9a16a640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb940b5320 0x7fcb940b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.919+0000 7fcb9a16a640 10 monclient(hunting): picked mon.noname-b con 0x7fcb940b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:56.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.919+0000 7fcb9a16a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcb940b1ac0 0x7fcb940b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.919+0000 7fcb9a16a640 10 monclient(hunting): picked mon.noname-c con 0x7fcb940b1ac0 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:56.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.919+0000 7fcb9a16a640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcb9413f630 0x7fcb9413fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.919+0000 7fcb9a16a640 10 monclient(hunting): picked mon.noname-a con 0x7fcb9413f630 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:56.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.919+0000 7fcb9a16a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.919+0000 7fcb9a16a640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb940b5700 con 0x7fcb9413f630 2024-03-13T07:34:56.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.919+0000 7fcb9a16a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.919+0000 7fcb9a16a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb940b1150 con 0x7fcb940b1ac0 2024-03-13T07:34:56.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.919+0000 7fcb9a16a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.919+0000 7fcb9a16a640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9413ff80 con 0x7fcb940b5320 2024-03-13T07:34:56.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.919+0000 7fcb9a16a640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:56.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.919+0000 7fcb9a16a640 10 monclient(hunting): authenticate will time out at 1324.327506s 2024-03-13T07:34:56.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.920+0000 7fcb937fe640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb940b5320 0x7fcb940b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.920+0000 7fcb92ffd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcb940b1ac0 0x7fcb940b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.920+0000 7fcb92ffd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcb940b1ac0 0x7fcb940b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:56116/0 (socket says 172.21.15.132:56116) 2024-03-13T07:34:56.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.920+0000 7fcb92ffd640 1 -- 172.21.15.132:0/1546193260 learned_addr learned my addr 172.21.15.132:0/1546193260 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:56.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.920+0000 7fcb92ffd640 10 monclient(hunting): get_auth_request con 0x7fcb940b1ac0 auth_method 0 2024-03-13T07:34:56.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.920+0000 7fcb92ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.920+0000 7fcb92ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.920+0000 7fcb92ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.920+0000 7fcb937fe640 10 monclient(hunting): get_auth_request con 0x7fcb940b5320 auth_method 0 2024-03-13T07:34:56.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.920+0000 7fcb937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.920+0000 7fcb937fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.920+0000 7fcb937fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.921+0000 7fcb92ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.921+0000 7fcb92ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.921 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.921+0000 7fcb92ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.921+0000 7fcb937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.921+0000 7fcb937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.921+0000 7fcb937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.922+0000 7fcb92ffd640 10 monclient(hunting): handle_auth_done global_id 4342 payload 274 2024-03-13T07:34:56.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.922+0000 7fcb92ffd640 1 -- 172.21.15.132:0/1546193260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb940b5320 msgr2=0x7fcb940b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:56.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.922+0000 7fcb92ffd640 1 --2- 172.21.15.132:0/1546193260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb940b5320 0x7fcb940b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.922+0000 7fcb92ffd640 1 -- 172.21.15.132:0/1546193260 >> v2:172.21.15.132:3300/0 conn(0x7fcb9413f630 msgr2=0x7fcb9413fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:56.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.922+0000 7fcb92ffd640 1 --2- 172.21.15.132:0/1546193260 >> v2:172.21.15.132:3300/0 conn(0x7fcb9413f630 0x7fcb9413fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.922+0000 7fcb92ffd640 10 monclient: _finish_hunting 0 2024-03-13T07:34:56.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.922+0000 7fcb92ffd640 1 monclient: found mon.noname-c 2024-03-13T07:34:56.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.922+0000 7fcb92ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:56.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.922+0000 7fcb92ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T07:34:56.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.922+0000 7fcb92ffd640 1 -- 172.21.15.132:0/1546193260 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb940b13c0 con 0x7fcb940b1ac0 2024-03-13T07:34:56.925 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-03-13T07:34:56.926 DEBUG:teuthology.parallel:result is None 2024-03-13T07:34:56.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.930+0000 7fcb92ffd640 1 --2- 172.21.15.132:0/1546193260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb940b1ac0 0x7fcb940b1f00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcb9406de20 tx=0x7fcb8802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9027852764921354 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:56.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.933+0000 7f3aafe0c640 10 monclient: get_monmap_and_config 2024-03-13T07:34:56.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.933+0000 7f3aafe0c640 10 monclient: build_initial_monmap 2024-03-13T07:34:56.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.933+0000 7f3aafe0c640 10 monclient: monmap: 2024-03-13T07:34:56.933 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:56.933 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.933 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:56.934537+0000 2024-03-13T07:34:56.933 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:56.934537+0000 2024-03-13T07:34:56.934 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:56.934 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.934 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:56.934 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:56.934 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:56.934 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:56.934 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:56.934 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.934+0000 7f3aafe0c640 1 Processor -- start 2024-03-13T07:34:56.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.934+0000 7f3aafe0c640 1 -- start start 2024-03-13T07:34:56.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.934+0000 7f3aafe0c640 10 monclient: init 2024-03-13T07:34:56.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.935+0000 7f3aafe0c640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:56.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.935+0000 7f3aafe0c640 10 monclient: _add_conns ranks=[0,1,2,4,3] 2024-03-13T07:34:56.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.936+0000 7f3aafe0c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3aa80b5320 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:56.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.936+0000 7f3aafe0c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3aa80b0220 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:56.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.936+0000 7f3aafe0c640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3aa813fd40 0x7f3aa814df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.936+0000 7f3aafe0c640 10 monclient(hunting): picked mon.noname-b con 0x7f3aa813fd40 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:56.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.936+0000 7f3aafe0c640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa80b1150 con 0x7f3aa80b5320 2024-03-13T07:34:56.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.936+0000 7f3aafe0c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa80b13c0 con 0x7f3aa80b0220 2024-03-13T07:34:56.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.936+0000 7f3aafe0c640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.936+0000 7f3aafe0c640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80b0690 con 0x7f3aa813fd40 2024-03-13T07:34:56.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.936+0000 7f3aafe0c640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:56.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.936+0000 7f3aafe0c640 10 monclient(hunting): authenticate will time out at 1324.344552s 2024-03-13T07:34:56.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.937+0000 7f3aae3a9640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3aa813fd40 0x7f3aa814df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.938+0000 7f3aae3a9640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3aa813fd40 0x7f3aa814df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:57848/0 (socket says 172.21.15.132:57848) 2024-03-13T07:34:56.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.938+0000 7f3aae3a9640 1 -- 172.21.15.132:0/1300264986 learned_addr learned my addr 172.21.15.132:0/1300264986 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:56.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.938+0000 7f3aae3a9640 10 monclient(hunting): get_auth_request con 0x7f3aa813fd40 auth_method 0 2024-03-13T07:34:56.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.938+0000 7f3aae3a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.938+0000 7f3aae3a9640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.938+0000 7f3aae3a9640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.939+0000 7f3aae3a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.939+0000 7f3aae3a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.939+0000 7f3aae3a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.941+0000 7f3aae3a9640 10 monclient(hunting): handle_auth_done global_id 4328 payload 274 2024-03-13T07:34:56.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.941+0000 7f3aae3a9640 1 -- 172.21.15.132:0/1300264986 >> v1:172.21.15.195:6789/0 conn(0x7f3aa80b0220 legacy=0x7f3aa813f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:56.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.941+0000 7f3aae3a9640 1 -- 172.21.15.132:0/1300264986 >> v1:172.21.15.132:6789/0 conn(0x7f3aa80b5320 legacy=0x7f3aa80b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:56.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.941+0000 7f3aae3a9640 10 monclient: _finish_hunting 0 2024-03-13T07:34:56.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.941+0000 7f3aae3a9640 1 monclient: found mon.noname-b 2024-03-13T07:34:56.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.941+0000 7f3aae3a9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:56.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.941+0000 7f3aae3a9640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T07:34:56.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.941+0000 7f3aae3a9640 1 -- 172.21.15.132:0/1300264986 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa80b16a0 con 0x7f3aa813fd40 2024-03-13T07:34:56.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.944+0000 7f3aae3a9640 1 --2- 172.21.15.132:0/1300264986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3aa813fd40 0x7f3aa814df00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009e80 tx=0x7f3aa002bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2a14442488ed935 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:56.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.949+0000 7f53625d8640 10 monclient: get_monmap_and_config 2024-03-13T07:34:56.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.949+0000 7f53625d8640 10 monclient: build_initial_monmap 2024-03-13T07:34:56.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.949+0000 7f53625d8640 10 monclient: monmap: 2024-03-13T07:34:56.949 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:56.950 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.950 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:56.950474+0000 2024-03-13T07:34:56.950 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:56.950474+0000 2024-03-13T07:34:56.950 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:56.950 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.950 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:56.950 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:56.950 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:56.950 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:56.950 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:56.950 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.950+0000 7f53625d8640 1 Processor -- start 2024-03-13T07:34:56.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.950+0000 7f53625d8640 1 -- start start 2024-03-13T07:34:56.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.950+0000 7f53625d8640 10 monclient: init 2024-03-13T07:34:56.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.951+0000 7f53625d8640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:56.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.951+0000 7f53625d8640 10 monclient: _add_conns ranks=[3,0,1,4,2] 2024-03-13T07:34:56.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.951+0000 7f53625d8640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f535c0b5320 0x7f535c0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.951+0000 7f53625d8640 10 monclient(hunting): picked mon.noname-a con 0x7f535c0b5320 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:56.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.951+0000 7f53625d8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f535c0b1ac0 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:56.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.951+0000 7f53625d8640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f535c13fd40 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:56.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.952+0000 7f53625d8640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535c0b1150 con 0x7f535c0b1ac0 2024-03-13T07:34:56.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.952+0000 7f53625d8640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535c0b13c0 con 0x7f535c13fd40 2024-03-13T07:34:56.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.952+0000 7f53625d8640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.952+0000 7f53625d8640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535c0b16a0 con 0x7f535c0b5320 2024-03-13T07:34:56.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.952+0000 7f53625d8640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:56.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.952+0000 7f53625d8640 10 monclient(hunting): authenticate will time out at 1324.360152s 2024-03-13T07:34:56.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.980+0000 7f535bfff640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f535c0b5320 0x7f535c0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.983+0000 7f535bfff640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f535c0b5320 0x7f535c0b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34628/0 (socket says 172.21.15.132:34628) 2024-03-13T07:34:56.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.983+0000 7f535bfff640 1 -- 172.21.15.132:0/3333789233 learned_addr learned my addr 172.21.15.132:0/3333789233 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:56.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.983+0000 7f535bfff640 10 monclient(hunting): get_auth_request con 0x7f535c0b5320 auth_method 0 2024-03-13T07:34:56.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.983+0000 7f535bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.983+0000 7f535bfff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.983+0000 7f535bfff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7f535bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7f535bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7f535bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.984 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb927fc640 1 -- 172.21.15.132:0/1546193260 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88031780 con 0x7fcb940b1ac0 2024-03-13T07:34:56.984 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.984 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb927fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:56.984 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb927fc640 10 monclient: dump: 2024-03-13T07:34:56.984 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.984 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.984 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.985 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.985 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.985 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.985 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.985 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.985 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.985 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb927fc640 10 monclient: _finish_auth 0 2024-03-13T07:34:56.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb927fc640 10 monclient: _check_auth_tickets 2024-03-13T07:34:56.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:56.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb927fc640 1 -- 172.21.15.132:0/1546193260 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb88037070 con 0x7fcb940b1ac0 2024-03-13T07:34:56.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb9a16a640 5 monclient: authenticate success, global_id 4342 2024-03-13T07:34:56.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb9a16a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:56.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb927fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:56.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb9a16a640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:56.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb9a16a640 4 set_mon_vals no callback set 2024-03-13T07:34:56.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb927fc640 1 -- 172.21.15.132:0/1546193260 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88031b80 con 0x7fcb940b1ac0 2024-03-13T07:34:56.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:56.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb927fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:56.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.984+0000 7fcb927fc640 10 monclient: dump: 2024-03-13T07:34:56.986 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:56.986 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:56.986 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.986 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:56.986 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:56.986 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.986 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:56.986 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:56.987 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:56.987 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.985+0000 7fcb9a16a640 10 monclient: shutdown 2024-03-13T07:34:56.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.985+0000 7fcb9a16a640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:56.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.985+0000 7fcb9a16a640 1 -- 172.21.15.132:0/1546193260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb940b1ac0 msgr2=0x7fcb940b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.985+0000 7fcb9a16a640 1 --2- 172.21.15.132:0/1546193260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb940b1ac0 0x7fcb940b1f00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcb9406de20 tx=0x7fcb8802b820 comp rx=0 tx=0).stop 2024-03-13T07:34:56.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.985+0000 7fcb9a16a640 1 -- 172.21.15.132:0/1546193260 shutdown_connections 2024-03-13T07:34:56.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.985+0000 7fcb9a16a640 1 --2- 172.21.15.132:0/1546193260 >> v2:172.21.15.132:3300/0 conn(0x7fcb9413f630 0x7fcb9413fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.985+0000 7fcb9a16a640 1 --2- 172.21.15.132:0/1546193260 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb940b1ac0 0x7fcb940b1f00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.985+0000 7fcb9a16a640 1 --2- 172.21.15.132:0/1546193260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb940b5320 0x7fcb940b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.985+0000 7fcb9a16a640 1 -- 172.21.15.132:0/1546193260 >> 172.21.15.132:0/1546193260 conn(0x7fcb9406d2b0 msgr2=0x7fcb9406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:56.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.985+0000 7fcb9a16a640 1 -- 172.21.15.132:0/1546193260 shutdown_connections 2024-03-13T07:34:56.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.985+0000 7fcb9a16a640 1 -- 172.21.15.132:0/1546193260 wait complete. 2024-03-13T07:34:56.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.986+0000 7fcb9a16a640 10 monclient: build_initial_monmap 2024-03-13T07:34:56.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.986+0000 7fcb9a16a640 10 monclient: monmap: 2024-03-13T07:34:56.989 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:56.990 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:56.990 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:56.990 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:56.990 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:56.990 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:56.990 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:56.990 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:56.990 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:56.990 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:56.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.986+0000 7fcb9a16a640 1 librados: starting msgr at 2024-03-13T07:34:56.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.986+0000 7fcb9a16a640 1 librados: starting objecter 2024-03-13T07:34:56.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.986+0000 7fcb9a16a640 1 Processor -- start 2024-03-13T07:34:56.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 1 -- start start 2024-03-13T07:34:56.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 1 librados: setting wanted keys 2024-03-13T07:34:56.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 1 librados: calling monclient init 2024-03-13T07:34:56.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 10 monclient: init 2024-03-13T07:34:56.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:56.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 10 monclient: _add_conns ranks=[2,1,0] 2024-03-13T07:34:56.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb940b1ac0 0x7fcb94145620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 10 monclient(hunting): picked mon.noname-c con 0x7fcb940b1ac0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:56.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb940b5320 0x7fcb94137740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 10 monclient(hunting): picked mon.noname-b con 0x7fcb940b5320 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:56.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb9413f630 0x7fcb94137c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:56.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 10 monclient(hunting): picked mon.noname-a con 0x7fcb9413f630 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:56.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb940b4800 con 0x7fcb9413f630 2024-03-13T07:34:56.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb940afd00 con 0x7fcb940b5320 2024-03-13T07:34:56.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:56.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb940b4980 con 0x7fcb940b1ac0 2024-03-13T07:34:56.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:56.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.987+0000 7fcb9a16a640 10 monclient(hunting): authenticate will time out at 1324.395807s 2024-03-13T07:34:56.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.988+0000 7f535bfff640 10 monclient(hunting): handle_auth_done global_id 4338 payload 274 2024-03-13T07:34:56.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.988+0000 7f535bfff640 1 -- 172.21.15.132:0/3333789233 >> v1:172.21.15.195:6789/0 conn(0x7f535c13fd40 legacy=0x7f535c14cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.988+0000 7f535bfff640 1 -- 172.21.15.132:0/3333789233 >> v1:172.21.15.132:6789/0 conn(0x7f535c0b1ac0 legacy=0x7f535c13f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:56.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.988+0000 7f535bfff640 10 monclient: _finish_hunting 0 2024-03-13T07:34:56.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.988+0000 7f535bfff640 1 monclient: found mon.noname-a 2024-03-13T07:34:56.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.988+0000 7f535bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:56.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.988+0000 7f535bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:56.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.988+0000 7f535bfff640 1 -- 172.21.15.132:0/3333789233 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c0afa20 con 0x7f535c0b5320 2024-03-13T07:34:56.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.991+0000 7fcb92ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb940b5320 0x7fcb94137740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:56.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.992+0000 7fcb92ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb940b5320 0x7fcb94137740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:56120/0 (socket says 172.21.15.132:56120) 2024-03-13T07:34:56.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.992+0000 7fcb92ffd640 1 -- 172.21.15.132:0/1283424030 learned_addr learned my addr 172.21.15.132:0/1283424030 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:56.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.992+0000 7fcb92ffd640 10 monclient(hunting): get_auth_request con 0x7fcb940b5320 auth_method 0 2024-03-13T07:34:56.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.992+0000 7fcb92ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:56.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.992+0000 7fcb92ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:56.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.992+0000 7fcb92ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:56.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.993+0000 7f535bfff640 1 --2- 172.21.15.132:0/3333789233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f535c0b5320 0x7f535c0b0220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f534c009870 tx=0x7f534c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b00a17c60d88c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:56.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.995+0000 7fcb92ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:56.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.995+0000 7fcb92ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:56.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.995+0000 7fcb92ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:56.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.999+0000 7fcb92ffd640 10 monclient(hunting): handle_auth_done global_id 4348 payload 824 2024-03-13T07:34:56.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.999+0000 7fcb92ffd640 1 -- 172.21.15.132:0/1283424030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb940b1ac0 msgr2=0x7fcb94145620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.998 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.999+0000 7fcb92ffd640 1 --2- 172.21.15.132:0/1283424030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb940b1ac0 0x7fcb94145620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.999+0000 7fcb92ffd640 1 -- 172.21.15.132:0/1283424030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb9413f630 msgr2=0x7fcb94137c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:56.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.999+0000 7fcb92ffd640 1 --2- 172.21.15.132:0/1283424030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb9413f630 0x7fcb94137c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:56.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.999+0000 7fcb92ffd640 10 monclient: _finish_hunting 0 2024-03-13T07:34:56.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.999+0000 7fcb92ffd640 1 monclient: found mon.noname-b 2024-03-13T07:34:56.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.999+0000 7fcb92ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:56.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.999+0000 7fcb92ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:56.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:56.999+0000 7fcb92ffd640 1 -- 172.21.15.132:0/1283424030 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb94142980 con 0x7fcb940b5320 2024-03-13T07:34:57.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.000+0000 7fdcda245640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2024-03-13T07:34:57.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.000+0000 7fdcda245640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2024-03-13T07:34:57.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.000+0000 7fdcda245640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:57.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.000+0000 7fdcda245640 1 -- 172.21.15.132:0/2313796398 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fdcd4067e10 con 0x7fdcd413f630 2024-03-13T07:34:57.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.002+0000 7fcb92ffd640 1 --2- 172.21.15.132:0/1283424030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb940b5320 0x7fcb94137740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcb88000c00 tx=0x7fcb880046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:57.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.002+0000 7fcb7bfff640 10 client.?.objecter ms_handle_connect 0x7fcb940b5320 2024-03-13T07:34:57.002 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.002+0000 7fcb7bfff640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:57.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb7bfff640 1 -- 172.21.15.132:0/1283424030 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88032610 con 0x7fcb940b5320 2024-03-13T07:34:57.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb7bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:57.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb7bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:57.007 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb7bfff640 10 monclient: dump: 2024-03-13T07:34:57.007 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:57.007 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:57.007 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.007 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.007 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:57.007 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.007 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:57.007 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:57.007 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:57.007 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.007 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb7bfff640 10 monclient: _finish_auth 0 2024-03-13T07:34:57.007 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb7bfff640 10 monclient: _check_auth_tickets 2024-03-13T07:34:57.007 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb7bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:57.007 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb7bfff640 1 -- 172.21.15.132:0/1283424030 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb880327d0 con 0x7fcb940b5320 2024-03-13T07:34:57.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb9a16a640 5 monclient: authenticate success, global_id 4348 2024-03-13T07:34:57.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb9a16a640 10 monclient: _renew_subs 2024-03-13T07:34:57.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb9a16a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:57.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb9a16a640 1 -- 172.21.15.132:0/1283424030 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb94143600 con 0x7fcb940b5320 2024-03-13T07:34:57.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb7bfff640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:57.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb7bfff640 1 -- 172.21.15.132:0/1283424030 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88032bc0 con 0x7fcb940b5320 2024-03-13T07:34:57.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb7bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:57.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb7bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:57.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb7bfff640 10 monclient: dump: 2024-03-13T07:34:57.008 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:57.008 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:57.008 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.008 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.009 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:57.009 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.009 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:57.009 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:57.009 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:57.009 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb9a16a640 10 client.4348.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:57.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb9a16a640 10 monclient: _renew_subs 2024-03-13T07:34:57.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb9a16a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:57.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb9a16a640 1 -- 172.21.15.132:0/1283424030 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb940ae370 con 0x7fcb940b5320 2024-03-13T07:34:57.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb9a16a640 1 librados: init done 2024-03-13T07:34:57.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.006+0000 7fcb91ffb640 4 set_mon_vals no callback set 2024-03-13T07:34:57.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.007+0000 7fdcbbfff640 1 -- 172.21.15.132:0/2313796398 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdccc05d490 con 0x7fdcd413f630 2024-03-13T07:34:57.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.007+0000 7fdcbbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2024-03-13T07:34:57.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.007+0000 7fdcbbfff640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:57.010 INFO:teuthology.orchestra.run.smithi132.stdout:64424509447 2024-03-13T07:34:57.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.007+0000 7fcb9a16a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:57.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.007+0000 7fcb9a16a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:57.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.007+0000 7fcb9a16a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:57.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.007+0000 7fcb9a16a640 1 -- 172.21.15.132:0/1283424030 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb940a9930 con 0x7fcb940b5320 2024-03-13T07:34:57.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aacba6640 1 -- 172.21.15.132:0/1300264986 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa002c930 con 0x7f3aa813fd40 2024-03-13T07:34:57.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aacba6640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:57.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aacba6640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:57.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aacba6640 10 monclient: dump: 2024-03-13T07:34:57.012 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:57.012 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:57.013 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.013 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.013 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:57.013 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.013 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:57.013 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:57.013 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:57.013 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aacba6640 10 monclient: _finish_auth 0 2024-03-13T07:34:57.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aacba6640 10 monclient: _check_auth_tickets 2024-03-13T07:34:57.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aacba6640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:57.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aacba6640 1 -- 172.21.15.132:0/1300264986 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa002cad0 con 0x7f3aa813fd40 2024-03-13T07:34:57.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aafe0c640 5 monclient: authenticate success, global_id 4328 2024-03-13T07:34:57.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aafe0c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:57.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aacba6640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:57.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aafe0c640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:57.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aafe0c640 4 set_mon_vals no callback set 2024-03-13T07:34:57.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aacba6640 1 -- 172.21.15.132:0/1300264986 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa00343f0 con 0x7f3aa813fd40 2024-03-13T07:34:57.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aacba6640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:57.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aacba6640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:57.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.008+0000 7f3aacba6640 10 monclient: dump: 2024-03-13T07:34:57.014 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:57.014 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:57.014 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.014 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.015 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:57.015 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.015 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:57.015 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:57.015 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:57.015 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.009+0000 7fdcda245640 10 librados: watch_flush enter 2024-03-13T07:34:57.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.009+0000 7fdcda245640 10 librados: watch_flush exit 2024-03-13T07:34:57.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.009+0000 7fdcda245640 20 client.4335.objecter shutdown clearing up homeless session... 2024-03-13T07:34:57.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.009+0000 7fdcda245640 10 client.4335.objecter successfully canceled tick 2024-03-13T07:34:57.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.009+0000 7fdcda245640 1 -- 172.21.15.132:0/2313796398 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fdcc006d560 msgr2=0x7fdcc006fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.009+0000 7fdcda245640 1 --2- 172.21.15.132:0/2313796398 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fdcc006d560 0x7fdcc006fa20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009710 tx=0x7fdcc4006c90 comp rx=0 tx=0).stop 2024-03-13T07:34:57.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.009+0000 7fdcda245640 10 monclient: shutdown 2024-03-13T07:34:57.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.009+0000 7fdcda245640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:57.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.009+0000 7fdcda245640 1 -- 172.21.15.132:0/2313796398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd413f630 msgr2=0x7fdcd41dafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.009+0000 7fdcda245640 1 --2- 172.21.15.132:0/2313796398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd413f630 0x7fdcd41dafe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00b970 tx=0x7fdccc00be30 comp rx=0 tx=0).stop 2024-03-13T07:34:57.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.009+0000 7f3aafe0c640 10 monclient: shutdown 2024-03-13T07:34:57.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.009+0000 7f3aafe0c640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:57.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.009+0000 7f3aafe0c640 1 -- 172.21.15.132:0/1300264986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3aa813fd40 msgr2=0x7f3aa814df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.009+0000 7f3aafe0c640 1 --2- 172.21.15.132:0/1300264986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3aa813fd40 0x7f3aa814df00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009e80 tx=0x7f3aa002bd90 comp rx=0 tx=0).stop 2024-03-13T07:34:57.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.010+0000 7fdcda245640 1 -- 172.21.15.132:0/2313796398 shutdown_connections 2024-03-13T07:34:57.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.010+0000 7fdcda245640 1 --2- 172.21.15.132:0/2313796398 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fdcc006d560 0x7fdcc006fa20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.018 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.010+0000 7fdcda245640 1 --2- 172.21.15.132:0/2313796398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd413f630 0x7fdcd41dafe0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.018 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.010+0000 7fdcda245640 1 --2- 172.21.15.132:0/2313796398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcd40b5320 0x7fdcd40ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.018 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.010+0000 7fdcda245640 1 --2- 172.21.15.132:0/2313796398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdcd40b1ac0 0x7fdcd40aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.018 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.010+0000 7fdcda245640 1 -- 172.21.15.132:0/2313796398 >> 172.21.15.132:0/2313796398 conn(0x7fdcd406d2b0 msgr2=0x7fdcd40b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:57.018 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.010+0000 7f3aafe0c640 1 -- 172.21.15.132:0/1300264986 shutdown_connections 2024-03-13T07:34:57.018 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.010+0000 7f3aafe0c640 1 --2- 172.21.15.132:0/1300264986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3aa813fd40 0x7f3aa814df00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.018 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.010+0000 7f3aafe0c640 1 -- 172.21.15.132:0/1300264986 >> 172.21.15.132:0/1300264986 conn(0x7f3aa806d2b0 msgr2=0x7f3aa806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:57.019 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.010+0000 7fdcda245640 1 -- 172.21.15.132:0/2313796398 shutdown_connections 2024-03-13T07:34:57.019 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.010+0000 7fdcda245640 1 -- 172.21.15.132:0/2313796398 wait complete. 2024-03-13T07:34:57.019 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.010+0000 7fdcda245640 1 librados: shutdown 2024-03-13T07:34:57.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.010+0000 7f3aafe0c640 1 -- 172.21.15.132:0/1300264986 shutdown_connections 2024-03-13T07:34:57.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.010+0000 7f3aafe0c640 1 -- 172.21.15.132:0/1300264986 wait complete. 2024-03-13T07:34:57.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.011+0000 7f3aafe0c640 10 monclient: build_initial_monmap 2024-03-13T07:34:57.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.011+0000 7f3aafe0c640 10 monclient: monmap: 2024-03-13T07:34:57.021 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:57.021 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:57.021 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:57.021 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:57.021 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:57.021 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.021 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:57.021 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:57.021 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:57.021 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f3aafe0c640 1 librados: starting msgr at 2024-03-13T07:34:57.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f3aafe0c640 1 librados: starting objecter 2024-03-13T07:34:57.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f3aafe0c640 1 Processor -- start 2024-03-13T07:34:57.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f3aafe0c640 1 -- start start 2024-03-13T07:34:57.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f3aafe0c640 1 librados: setting wanted keys 2024-03-13T07:34:57.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f3aafe0c640 1 librados: calling monclient init 2024-03-13T07:34:57.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f3aafe0c640 10 monclient: init 2024-03-13T07:34:57.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f535affd640 1 -- 172.21.15.132:0/3333789233 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c02e070 con 0x7f535c0b5320 2024-03-13T07:34:57.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f535affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:57.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f535affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:57.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f535affd640 10 monclient: dump: 2024-03-13T07:34:57.023 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:57.023 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:57.023 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.023 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.023 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:57.023 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.023 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:57.023 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:57.023 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:57.023 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f535affd640 10 monclient: _finish_auth 0 2024-03-13T07:34:57.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f535affd640 10 monclient: _check_auth_tickets 2024-03-13T07:34:57.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f535affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:57.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f535affd640 1 -- 172.21.15.132:0/3333789233 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c037070 con 0x7f535c0b5320 2024-03-13T07:34:57.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f53625d8640 5 monclient: authenticate success, global_id 4338 2024-03-13T07:34:57.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f53625d8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:57.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f535affd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:57.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f53625d8640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:57.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f535affd640 1 -- 172.21.15.132:0/3333789233 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c031a00 con 0x7f535c0b5320 2024-03-13T07:34:57.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f53625d8640 4 set_mon_vals no callback set 2024-03-13T07:34:57.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f535affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:57.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f535affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:57.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.012+0000 7f535affd640 10 monclient: dump: 2024-03-13T07:34:57.024 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:57.024 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:57.024 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.024 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.025 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:57.025 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.025 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:57.025 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:57.025 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:57.025 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.013+0000 7f3aafe0c640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:57.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.013+0000 7f3aafe0c640 10 monclient: _add_conns ranks=[2,0,1] 2024-03-13T07:34:57.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.013+0000 7f3aafe0c640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3aa80b0220 0x7f3aa8142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:57.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.013+0000 7f3aafe0c640 10 monclient(hunting): picked mon.noname-c con 0x7f3aa80b0220 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:57.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.013+0000 7f3aafe0c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3aa80b5320 0x7f3aa8145280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:57.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.013+0000 7f3aafe0c640 10 monclient(hunting): picked mon.noname-a con 0x7f3aa80b5320 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:57.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.013+0000 7f53625d8640 10 monclient: shutdown 2024-03-13T07:34:57.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.013+0000 7f53625d8640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:57.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.013+0000 7f53625d8640 1 -- 172.21.15.132:0/3333789233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f535c0b5320 msgr2=0x7f535c0b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.013+0000 7f53625d8640 1 --2- 172.21.15.132:0/3333789233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f535c0b5320 0x7f535c0b0220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f534c009870 tx=0x7f534c02b880 comp rx=0 tx=0).stop 2024-03-13T07:34:57.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.014+0000 7f53625d8640 1 -- 172.21.15.132:0/3333789233 shutdown_connections 2024-03-13T07:34:57.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.014+0000 7f53625d8640 1 --2- 172.21.15.132:0/3333789233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f535c0b5320 0x7f535c0b0220 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.014+0000 7f53625d8640 1 -- 172.21.15.132:0/3333789233 >> 172.21.15.132:0/3333789233 conn(0x7f535c06d2b0 msgr2=0x7f535c06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:57.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.014+0000 7f53625d8640 1 -- 172.21.15.132:0/3333789233 shutdown_connections 2024-03-13T07:34:57.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.014+0000 7f3aafe0c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3aa813fd40 0x7f3aa8145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:57.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.014+0000 7f53625d8640 1 -- 172.21.15.132:0/3333789233 wait complete. 2024-03-13T07:34:57.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.014+0000 7f3aafe0c640 10 monclient(hunting): picked mon.noname-b con 0x7f3aa813fd40 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:57.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.014+0000 7f3aadba8640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3aa80b0220 0x7f3aa8142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:57.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.014+0000 7f3aafe0c640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:57.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.014+0000 7f3aafe0c640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80b4800 con 0x7f3aa80b5320 2024-03-13T07:34:57.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.014+0000 7f3aafe0c640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:57.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.014+0000 7f3aafe0c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80afd00 con 0x7f3aa813fd40 2024-03-13T07:34:57.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.014+0000 7f3aafe0c640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:57.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.014+0000 7f3aafe0c640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80b4980 con 0x7f3aa80b0220 2024-03-13T07:34:57.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.014+0000 7f3aadba8640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3aa80b0220 0x7f3aa8142aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:57860/0 (socket says 172.21.15.132:57860) 2024-03-13T07:34:57.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.014+0000 7f3aadba8640 1 -- 172.21.15.132:0/229728910 learned_addr learned my addr 172.21.15.132:0/229728910 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:57.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.015+0000 7f3aafe0c640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:57.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.015+0000 7f3aafe0c640 10 monclient(hunting): authenticate will time out at 1324.423150s 2024-03-13T07:34:57.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.015+0000 7f3aadba8640 10 monclient(hunting): get_auth_request con 0x7f3aa80b0220 auth_method 0 2024-03-13T07:34:57.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.015+0000 7f3aadba8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:57.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.015+0000 7f3aadba8640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:57.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.015+0000 7f3aadba8640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:57.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.015+0000 7f53625d8640 10 monclient: build_initial_monmap 2024-03-13T07:34:57.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.015+0000 7f53625d8640 10 monclient: monmap: 2024-03-13T07:34:57.032 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:57.032 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:57.032 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:57.032 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:57.032 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:57.032 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.032 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:57.032 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:57.032 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:57.032 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.015+0000 7f3aadba8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:57.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.015+0000 7f3aadba8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:57.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.015+0000 7f3aadba8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:57.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.015+0000 7f53625d8640 1 librados: starting msgr at 2024-03-13T07:34:57.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.015+0000 7f53625d8640 1 librados: starting objecter 2024-03-13T07:34:57.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.016+0000 7f53625d8640 1 Processor -- start 2024-03-13T07:34:57.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.016+0000 7f53625d8640 1 -- start start 2024-03-13T07:34:57.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.016+0000 7f53625d8640 1 librados: setting wanted keys 2024-03-13T07:34:57.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.016+0000 7f53625d8640 1 librados: calling monclient init 2024-03-13T07:34:57.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.016+0000 7f53625d8640 10 monclient: init 2024-03-13T07:34:57.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.017+0000 7f53625d8640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:57.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.017+0000 7f53625d8640 10 monclient: _add_conns ranks=[1,0,2] 2024-03-13T07:34:57.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.017+0000 7f53625d8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f535c0b1ac0 0x7f535c1d82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:57.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.017+0000 7f53625d8640 10 monclient(hunting): picked mon.noname-b con 0x7f535c0b1ac0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:57.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.017+0000 7f53625d8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f535c0b5320 0x7f535c1d8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:57.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.017+0000 7f53625d8640 10 monclient(hunting): picked mon.noname-a con 0x7f535c0b5320 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:57.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.017+0000 7f53625d8640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f535c13fd40 0x7f535c0a9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:57.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.017+0000 7f53625d8640 10 monclient(hunting): picked mon.noname-c con 0x7f535c13fd40 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:57.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.017+0000 7f53625d8640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:57.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.017+0000 7f53625d8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c0b4980 con 0x7f535c0b5320 2024-03-13T07:34:57.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.017+0000 7f53625d8640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:57.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.017+0000 7f53625d8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c0b4800 con 0x7f535c0b1ac0 2024-03-13T07:34:57.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.017+0000 7f53625d8640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:57.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.017+0000 7f53625d8640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c0b1f30 con 0x7f535c13fd40 2024-03-13T07:34:57.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.017+0000 7f53625d8640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:57.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.017+0000 7f53625d8640 10 monclient(hunting): authenticate will time out at 1324.425909s 2024-03-13T07:34:57.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.019+0000 7f3aadba8640 10 monclient(hunting): handle_auth_done global_id 4331 payload 824 2024-03-13T07:34:57.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.019+0000 7f3aadba8640 1 -- 172.21.15.132:0/229728910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3aa813fd40 msgr2=0x7f3aa8145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.019+0000 7f3aadba8640 1 --2- 172.21.15.132:0/229728910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3aa813fd40 0x7f3aa8145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.019+0000 7f3aadba8640 1 -- 172.21.15.132:0/229728910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3aa80b5320 msgr2=0x7f3aa8145280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.019+0000 7f3aadba8640 1 --2- 172.21.15.132:0/229728910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3aa80b5320 0x7f3aa8145280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.019+0000 7f3aadba8640 10 monclient: _finish_hunting 0 2024-03-13T07:34:57.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.019+0000 7f3aadba8640 1 monclient: found mon.noname-c 2024-03-13T07:34:57.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.019+0000 7f3aadba8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:57.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.019+0000 7f3aadba8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:34:57.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.019+0000 7f3aadba8640 1 -- 172.21.15.132:0/229728910 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa81cc100 con 0x7f3aa80b0220 2024-03-13T07:34:57.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.021+0000 7f5360b75640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f535c13fd40 0x7f535c0a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:57.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.022+0000 7f3aadba8640 1 --2- 172.21.15.132:0/229728910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3aa80b0220 0x7f3aa8142aa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400c410 tx=0x7f3aa400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:57.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.022+0000 7f3a9e7fc640 10 client.?.objecter ms_handle_connect 0x7f3aa80b0220 2024-03-13T07:34:57.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.022+0000 7f3a9e7fc640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:57.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.022+0000 7f5360b75640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f535c13fd40 0x7f535c0a9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:57872/0 (socket says 172.21.15.132:57872) 2024-03-13T07:34:57.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.022+0000 7f5360b75640 1 -- 172.21.15.132:0/2715453845 learned_addr learned my addr 172.21.15.132:0/2715453845 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:57.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.022+0000 7f5360b75640 10 monclient(hunting): get_auth_request con 0x7f535c13fd40 auth_method 0 2024-03-13T07:34:57.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.022+0000 7f5360b75640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:57.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.022+0000 7f5360b75640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:57.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.022+0000 7f5360b75640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:57.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.022+0000 7f5360b75640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:57.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.022+0000 7f5360b75640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:57.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.023+0000 7f5360b75640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:57.042 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2024-03-13T07:34:57.042 DEBUG:teuthology.parallel:result is None 2024-03-13T07:34:57.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.026+0000 7fcb7bfff640 1 -- 172.21.15.132:0/1283424030 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fcb880453f0 con 0x7fcb940b5320 2024-03-13T07:34:57.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.026+0000 7fcb7bfff640 1 --2- 172.21.15.132:0/1283424030 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcb8006d5b0 0x7fcb8006fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:57.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.026+0000 7fcb937fe640 1 --2- 172.21.15.132:0/1283424030 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcb8006d5b0 0x7fcb8006fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:57.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.026+0000 7fcb937fe640 10 monclient: get_auth_request con 0x7fcb8006d5b0 auth_method 0 2024-03-13T07:34:57.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.026+0000 7f535b7fe640 1 --2- 172.21.15.132:0/2715453845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f535c0b5320 0x7f535c1d8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:57.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.027+0000 7fcb7bfff640 10 client.4348.objecter ms_handle_connect 0x7fcb8006d5b0 2024-03-13T07:34:57.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.027+0000 7fcb937fe640 1 --2- 172.21.15.132:0/1283424030 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcb8006d5b0 0x7fcb8006fa70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb84006a90 tx=0x7fcb84006a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:57.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.027+0000 7f5360b75640 10 monclient(hunting): handle_auth_done global_id 4334 payload 824 2024-03-13T07:34:57.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.027+0000 7f5360b75640 1 -- 172.21.15.132:0/2715453845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f535c0b1ac0 msgr2=0x7f535c1d82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.027+0000 7f5360b75640 1 --2- 172.21.15.132:0/2715453845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f535c0b1ac0 0x7f535c1d82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.027+0000 7f5360b75640 1 -- 172.21.15.132:0/2715453845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f535c0b5320 msgr2=0x7f535c1d8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.027+0000 7f5360b75640 1 --2- 172.21.15.132:0/2715453845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f535c0b5320 0x7f535c1d8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.027+0000 7f5360b75640 10 monclient: _finish_hunting 0 2024-03-13T07:34:57.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.027+0000 7f5360b75640 1 monclient: found mon.noname-c 2024-03-13T07:34:57.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.027+0000 7f5360b75640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:57.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.027+0000 7f5360b75640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:34:57.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.027+0000 7f5360b75640 1 -- 172.21.15.132:0/2715453845 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c146990 con 0x7f535c13fd40 2024-03-13T07:34:57.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.028+0000 7fcb7bfff640 1 -- 172.21.15.132:0/1283424030 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcb880792b0 con 0x7fcb940b5320 2024-03-13T07:34:57.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.028+0000 7fcb7bfff640 10 client.4348.objecter ms_dispatch 0x7fcb94000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:57.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.028+0000 7fcb7bfff640 3 client.4348.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:57.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.028+0000 7fcb7bfff640 3 client.4348.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:57.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.028+0000 7fcb7bfff640 20 client.4348.objecter dump_active .. 0 homeless 2024-03-13T07:34:57.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.031+0000 7f3a9e7fc640 1 -- 172.21.15.132:0/229728910 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4017070 con 0x7f3aa80b0220 2024-03-13T07:34:57.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.031+0000 7f3a9e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:57.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.031+0000 7f3a9e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:57.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.031+0000 7f3a9e7fc640 10 monclient: dump: 2024-03-13T07:34:57.047 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:57.047 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:57.047 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.047 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.047 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:57.047 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.047 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:57.047 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:57.047 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:57.047 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.031+0000 7f3a9e7fc640 10 monclient: _finish_auth 0 2024-03-13T07:34:57.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.031+0000 7f3a9e7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:34:57.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.031+0000 7f3a9e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:57.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.031+0000 7f3a9e7fc640 1 -- 172.21.15.132:0/229728910 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa40035f0 con 0x7f3aa80b0220 2024-03-13T07:34:57.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.031+0000 7f3a9e7fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:57.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.032+0000 7f3a9e7fc640 1 -- 172.21.15.132:0/229728910 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4003a20 con 0x7f3aa80b0220 2024-03-13T07:34:57.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.032+0000 7f3a9e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:57.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.032+0000 7f3a9e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:57.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.032+0000 7f3a9e7fc640 10 monclient: dump: 2024-03-13T07:34:57.048 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:57.048 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:57.048 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.048 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.048 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:57.048 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.049 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:57.049 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:57.049 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:57.049 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.032+0000 7f3a9ffff640 4 set_mon_vals no callback set 2024-03-13T07:34:57.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.032+0000 7f3aafe0c640 5 monclient: authenticate success, global_id 4331 2024-03-13T07:34:57.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.032+0000 7f3aafe0c640 10 monclient: _renew_subs 2024-03-13T07:34:57.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.032+0000 7f3aafe0c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:57.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.032+0000 7f3aafe0c640 1 -- 172.21.15.132:0/229728910 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa81cc370 con 0x7f3aa80b0220 2024-03-13T07:34:57.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.032+0000 7f3aafe0c640 10 client.4331.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:57.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.032+0000 7f3aafe0c640 10 monclient: _renew_subs 2024-03-13T07:34:57.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.032+0000 7f3aafe0c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:57.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.032+0000 7f3aafe0c640 1 -- 172.21.15.132:0/229728910 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa81d7ca0 con 0x7f3aa80b0220 2024-03-13T07:34:57.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.032+0000 7f3aafe0c640 1 librados: init done 2024-03-13T07:34:57.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.033+0000 7f5360b75640 1 --2- 172.21.15.132:0/2715453845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f535c13fd40 0x7f535c0a9740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f535400c730 tx=0x7f535400cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:57.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.033+0000 7f5358ff9640 10 client.?.objecter ms_handle_connect 0x7f535c13fd40 2024-03-13T07:34:57.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.033+0000 7f5358ff9640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:57.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.033+0000 7f3aafe0c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:57.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.033+0000 7f3aafe0c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:57.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.033+0000 7f3aafe0c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:57.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.033+0000 7f3aafe0c640 1 -- 172.21.15.132:0/229728910 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa804f000 con 0x7f3aa80b0220 2024-03-13T07:34:57.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.053+0000 7f3a9e7fc640 1 -- 172.21.15.132:0/229728910 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f3aa401a070 con 0x7f3aa80b0220 2024-03-13T07:34:57.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.054+0000 7f3a9e7fc640 1 --2- 172.21.15.132:0/229728910 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3a8c06d5b0 0x7f3a8c06fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:57.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.054+0000 7f3aad3a7640 1 --2- 172.21.15.132:0/229728910 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3a8c06d5b0 0x7f3a8c06fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:57.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.054+0000 7f3aad3a7640 10 monclient: get_auth_request con 0x7f3a8c06d5b0 auth_method 0 2024-03-13T07:34:57.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.054+0000 7f3a9e7fc640 10 client.4331.objecter ms_handle_connect 0x7f3a8c06d5b0 2024-03-13T07:34:57.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.054+0000 7f3aad3a7640 1 --2- 172.21.15.132:0/229728910 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3a8c06d5b0 0x7f3a8c06fa70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3aa81cf7f0 tx=0x7f3a98007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:57.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.107+0000 7f19f3c47640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-03-13T07:34:57.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.107+0000 7f19f3c47640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-03-13T07:34:57.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.107+0000 7f19f3c47640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:57.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.107+0000 7f19f3c47640 1 -- 172.21.15.132:0/1537107393 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f19ec04f000 con 0x7f19ec0b1ac0 2024-03-13T07:34:57.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.277+0000 7fcb7bfff640 1 -- 172.21.15.132:0/1283424030 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fcb88053080 con 0x7fcb940b5320 2024-03-13T07:34:57.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.277+0000 7fcb7bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:57.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.277+0000 7fcb7bfff640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:57.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.299+0000 7f3a9e7fc640 1 -- 172.21.15.132:0/229728910 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3aa4056870 con 0x7f3aa80b0220 2024-03-13T07:34:57.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.299+0000 7f3a9e7fc640 10 client.4331.objecter ms_dispatch 0x7f3aa8000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:57.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.299+0000 7f3a9e7fc640 3 client.4331.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:57.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.299+0000 7f3a9e7fc640 3 client.4331.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:57.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.299+0000 7f3a9e7fc640 20 client.4331.objecter dump_active .. 0 homeless 2024-03-13T07:34:57.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.299+0000 7f3a9e7fc640 1 -- 172.21.15.132:0/229728910 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f3aa4021020 con 0x7f3aa80b0220 2024-03-13T07:34:57.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.299+0000 7f3a9e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:57.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.299+0000 7f3a9e7fc640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:57.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f5358ff9640 1 -- 172.21.15.132:0/2715453845 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354016020 con 0x7f535c13fd40 2024-03-13T07:34:57.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f5358ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:57.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f5358ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:57.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f5358ff9640 10 monclient: dump: 2024-03-13T07:34:57.305 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:57.305 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:57.305 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.305 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.305 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:57.305 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.305 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:57.305 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:57.305 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:57.305 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f5358ff9640 10 monclient: _finish_auth 0 2024-03-13T07:34:57.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f5358ff9640 10 monclient: _check_auth_tickets 2024-03-13T07:34:57.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f5358ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:57.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f5358ff9640 1 -- 172.21.15.132:0/2715453845 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5354006d50 con 0x7f535c13fd40 2024-03-13T07:34:57.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f53625d8640 5 monclient: authenticate success, global_id 4334 2024-03-13T07:34:57.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f53625d8640 10 monclient: _renew_subs 2024-03-13T07:34:57.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f53625d8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:57.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f53625d8640 1 -- 172.21.15.132:0/2715453845 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c0ab650 con 0x7f535c13fd40 2024-03-13T07:34:57.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f5358ff9640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:57.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f5358ff9640 1 -- 172.21.15.132:0/2715453845 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354003bb0 con 0x7f535c13fd40 2024-03-13T07:34:57.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f5358ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:57.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f5358ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:57.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f5358ff9640 10 monclient: dump: 2024-03-13T07:34:57.306 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:57.307 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:57.307 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.307 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.307 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:57.307 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.307 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:57.307 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:57.307 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:57.307 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.304+0000 7f535a7fc640 4 set_mon_vals no callback set 2024-03-13T07:34:57.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.305+0000 7f53625d8640 10 client.4334.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:57.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.305+0000 7f53625d8640 10 monclient: _renew_subs 2024-03-13T07:34:57.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.305+0000 7f53625d8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:57.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.305+0000 7f53625d8640 1 -- 172.21.15.132:0/2715453845 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535c143fb0 con 0x7f535c13fd40 2024-03-13T07:34:57.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.305+0000 7f53625d8640 1 librados: init done 2024-03-13T07:34:57.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.305+0000 7f53625d8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:57.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.306+0000 7f53625d8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:57.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.306+0000 7f53625d8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:57.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.306+0000 7f53625d8640 1 -- 172.21.15.132:0/2715453845 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535c04f000 con 0x7f535c13fd40 2024-03-13T07:34:57.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.308+0000 7f19e27fc640 1 -- 172.21.15.132:0/1537107393 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f19e807f0a0 con 0x7f19ec0b1ac0 2024-03-13T07:34:57.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.308+0000 7f19e27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-03-13T07:34:57.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.308+0000 7f19e27fc640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:57.309 INFO:teuthology.orchestra.run.smithi132.stdout:85899345927 2024-03-13T07:34:57.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.310+0000 7f19f3c47640 10 librados: watch_flush enter 2024-03-13T07:34:57.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.310+0000 7f19f3c47640 10 librados: watch_flush exit 2024-03-13T07:34:57.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.310+0000 7f19f3c47640 20 client.4339.objecter shutdown clearing up homeless session... 2024-03-13T07:34:57.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.310+0000 7f19f3c47640 10 client.4339.objecter successfully canceled tick 2024-03-13T07:34:57.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.311+0000 7f19f3c47640 1 -- 172.21.15.132:0/1537107393 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f19d006d5b0 msgr2=0x7f19d006fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.311+0000 7f19f3c47640 1 --2- 172.21.15.132:0/1537107393 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f19d006d5b0 0x7f19d006fa70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0b3f40 tx=0x7f19dc009040 comp rx=0 tx=0).stop 2024-03-13T07:34:57.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.311+0000 7f19f3c47640 10 monclient: shutdown 2024-03-13T07:34:57.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.311+0000 7f19f3c47640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:57.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.311+0000 7f19f3c47640 1 -- 172.21.15.132:0/1537107393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19ec0b1ac0 msgr2=0x7f19ec146640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.311+0000 7f19f3c47640 1 --2- 172.21.15.132:0/1537107393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19ec0b1ac0 0x7f19ec146640 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f19e8000c00 tx=0x7f19e80046c0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.311+0000 7f19f3c47640 1 -- 172.21.15.132:0/1537107393 shutdown_connections 2024-03-13T07:34:57.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.311+0000 7f19f3c47640 1 --2- 172.21.15.132:0/1537107393 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f19d006d5b0 0x7f19d006fa70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.311+0000 7f19f3c47640 1 --2- 172.21.15.132:0/1537107393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19ec13fd40 0x7f19ec143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.311+0000 7f19f3c47640 1 --2- 172.21.15.132:0/1537107393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19ec0b5320 0x7f19ec145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.311+0000 7f19f3c47640 1 --2- 172.21.15.132:0/1537107393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19ec0b1ac0 0x7f19ec146640 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.311+0000 7f19f3c47640 1 -- 172.21.15.132:0/1537107393 >> 172.21.15.132:0/1537107393 conn(0x7f19ec06d2b0 msgr2=0x7f19ec13e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:57.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.311+0000 7f19f3c47640 1 -- 172.21.15.132:0/1537107393 shutdown_connections 2024-03-13T07:34:57.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.311+0000 7f19f3c47640 1 -- 172.21.15.132:0/1537107393 wait complete. 2024-03-13T07:34:57.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.311+0000 7f19f3c47640 1 librados: shutdown 2024-03-13T07:34:57.321 INFO:tasks.ceph.ceph_manager.ceph:need seq 85899345927 got 85899345927 for osd.2 2024-03-13T07:34:57.322 DEBUG:teuthology.parallel:result is None 2024-03-13T07:34:57.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.478+0000 7fcb9a16a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-03-13T07:34:57.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.478+0000 7fcb9a16a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-03-13T07:34:57.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.478+0000 7fcb9a16a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:57.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.478+0000 7fcb9a16a640 1 -- 172.21.15.132:0/1283424030 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcb9404f000 con 0x7fcb940b5320 2024-03-13T07:34:57.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.482+0000 7fcb7bfff640 1 -- 172.21.15.132:0/1283424030 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcb88040070 con 0x7fcb940b5320 2024-03-13T07:34:57.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.482+0000 7fcb7bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-03-13T07:34:57.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.482+0000 7fcb7bfff640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:57.482 INFO:teuthology.orchestra.run.smithi132.stdout:68719476742 2024-03-13T07:34:57.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 10 librados: watch_flush enter 2024-03-13T07:34:57.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 10 librados: watch_flush exit 2024-03-13T07:34:57.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 20 client.4348.objecter shutdown clearing up homeless session... 2024-03-13T07:34:57.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 10 client.4348.objecter successfully canceled tick 2024-03-13T07:34:57.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 1 -- 172.21.15.132:0/1283424030 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcb8006d5b0 msgr2=0x7fcb8006fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 1 --2- 172.21.15.132:0/1283424030 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcb8006d5b0 0x7fcb8006fa70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb84006a90 tx=0x7fcb84006a20 comp rx=0 tx=0).stop 2024-03-13T07:34:57.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 10 monclient: shutdown 2024-03-13T07:34:57.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:57.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 1 -- 172.21.15.132:0/1283424030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb940b5320 msgr2=0x7fcb94137740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 1 --2- 172.21.15.132:0/1283424030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb940b5320 0x7fcb94137740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcb88000c00 tx=0x7fcb880046c0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 1 -- 172.21.15.132:0/1283424030 shutdown_connections 2024-03-13T07:34:57.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 1 --2- 172.21.15.132:0/1283424030 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fcb8006d5b0 0x7fcb8006fa70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 1 --2- 172.21.15.132:0/1283424030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb9413f630 0x7fcb94137c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 1 --2- 172.21.15.132:0/1283424030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb940b5320 0x7fcb94137740 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 1 --2- 172.21.15.132:0/1283424030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcb940b1ac0 0x7fcb94145620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 1 -- 172.21.15.132:0/1283424030 >> 172.21.15.132:0/1283424030 conn(0x7fcb9406d2b0 msgr2=0x7fcb9413b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:57.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 1 -- 172.21.15.132:0/1283424030 shutdown_connections 2024-03-13T07:34:57.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 1 -- 172.21.15.132:0/1283424030 wait complete. 2024-03-13T07:34:57.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.484+0000 7fcb9a16a640 1 librados: shutdown 2024-03-13T07:34:57.495 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476742 for osd.3 2024-03-13T07:34:57.495 DEBUG:teuthology.parallel:result is None 2024-03-13T07:34:57.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.523+0000 7f3aafe0c640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-03-13T07:34:57.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.523+0000 7f3aafe0c640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-03-13T07:34:57.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.523+0000 7f3aafe0c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:57.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.523+0000 7f3aafe0c640 1 -- 172.21.15.132:0/229728910 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3aa813e920 con 0x7f3aa80b0220 2024-03-13T07:34:57.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.596+0000 7f5358ff9640 1 -- 172.21.15.132:0/2715453845 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f53540050d0 con 0x7f535c13fd40 2024-03-13T07:34:57.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.596+0000 7f5358ff9640 1 --2- 172.21.15.132:0/2715453845 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f534806d590 0x7f534806fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:57.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.596+0000 7f535bfff640 1 --2- 172.21.15.132:0/2715453845 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f534806d590 0x7f534806fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:57.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.596+0000 7f535bfff640 10 monclient: get_auth_request con 0x7f534806d590 auth_method 0 2024-03-13T07:34:57.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.596+0000 7f535bfff640 1 --2- 172.21.15.132:0/2715453845 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f534806d590 0x7f534806fa50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f534c036000 tx=0x7f534c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:57.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.596+0000 7f5358ff9640 10 client.4334.objecter ms_handle_connect 0x7f534806d590 2024-03-13T07:34:57.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.610+0000 7f5358ff9640 1 -- 172.21.15.132:0/2715453845 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5354057960 con 0x7f535c13fd40 2024-03-13T07:34:57.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.610+0000 7f5358ff9640 10 client.4334.objecter ms_dispatch 0x7f535c000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:57.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.610+0000 7f5358ff9640 3 client.4334.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:57.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.610+0000 7f5358ff9640 3 client.4334.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:57.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.610+0000 7f5358ff9640 20 client.4334.objecter dump_active .. 0 homeless 2024-03-13T07:34:57.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.610+0000 7f5358ff9640 1 -- 172.21.15.132:0/2715453845 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f5354020020 con 0x7f535c13fd40 2024-03-13T07:34:57.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.610+0000 7f5358ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:57.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.610+0000 7f5358ff9640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:57.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.615+0000 7f3a9e7fc640 1 -- 172.21.15.132:0/229728910 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3aa4058ed0 con 0x7f3aa80b0220 2024-03-13T07:34:57.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.615+0000 7f3a9e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-03-13T07:34:57.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.615+0000 7f3a9e7fc640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:57.615 INFO:teuthology.orchestra.run.smithi132.stdout:85899345930 2024-03-13T07:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.617+0000 7f3aafe0c640 10 librados: watch_flush enter 2024-03-13T07:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.617+0000 7f3aafe0c640 10 librados: watch_flush exit 2024-03-13T07:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.617+0000 7f3aafe0c640 20 client.4331.objecter shutdown clearing up homeless session... 2024-03-13T07:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.617+0000 7f3aafe0c640 10 client.4331.objecter successfully canceled tick 2024-03-13T07:34:57.618 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.617+0000 7f3aafe0c640 1 -- 172.21.15.132:0/229728910 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3a8c06d5b0 msgr2=0x7f3a8c06fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.617+0000 7f3aafe0c640 1 --2- 172.21.15.132:0/229728910 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3a8c06d5b0 0x7f3a8c06fa70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3aa81cf7f0 tx=0x7f3a98007990 comp rx=0 tx=0).stop 2024-03-13T07:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.617+0000 7f3aafe0c640 10 monclient: shutdown 2024-03-13T07:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.617+0000 7f3aafe0c640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.617+0000 7f3aafe0c640 1 -- 172.21.15.132:0/229728910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3aa80b0220 msgr2=0x7f3aa8142aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.617+0000 7f3aafe0c640 1 --2- 172.21.15.132:0/229728910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3aa80b0220 0x7f3aa8142aa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400c410 tx=0x7f3aa400c8d0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.618+0000 7f3aafe0c640 1 -- 172.21.15.132:0/229728910 shutdown_connections 2024-03-13T07:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.618+0000 7f3aafe0c640 1 --2- 172.21.15.132:0/229728910 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3aa813fd40 0x7f3aa8145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.618+0000 7f3aafe0c640 1 --2- 172.21.15.132:0/229728910 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f3a8c06d5b0 0x7f3a8c06fa70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.618+0000 7f3aafe0c640 1 --2- 172.21.15.132:0/229728910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3aa80b5320 0x7f3aa8145280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.618+0000 7f3aafe0c640 1 --2- 172.21.15.132:0/229728910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3aa80b0220 0x7f3aa8142aa0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.618+0000 7f3aafe0c640 1 -- 172.21.15.132:0/229728910 >> 172.21.15.132:0/229728910 conn(0x7f3aa806d2b0 msgr2=0x7f3aa813f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.618+0000 7f3aafe0c640 1 -- 172.21.15.132:0/229728910 shutdown_connections 2024-03-13T07:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.618+0000 7f3aafe0c640 1 -- 172.21.15.132:0/229728910 wait complete. 2024-03-13T07:34:57.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.618+0000 7f3aafe0c640 1 librados: shutdown 2024-03-13T07:34:57.629 INFO:tasks.ceph.ceph_manager.ceph:need seq 85899345929 got 85899345930 for osd.1 2024-03-13T07:34:57.630 DEBUG:teuthology.parallel:result is None 2024-03-13T07:34:57.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.786+0000 7f53625d8640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2024-03-13T07:34:57.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.786+0000 7f53625d8640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2024-03-13T07:34:57.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.786+0000 7f53625d8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:57.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.786+0000 7f53625d8640 1 -- 172.21.15.132:0/2715453845 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f535c13e6e0 con 0x7f535c13fd40 2024-03-13T07:34:57.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.790+0000 7f5358ff9640 1 -- 172.21.15.132:0/2715453845 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5354059fc0 con 0x7f535c13fd40 2024-03-13T07:34:57.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.790+0000 7f5358ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2024-03-13T07:34:57.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.790+0000 7f5358ff9640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:57.790 INFO:teuthology.orchestra.run.smithi132.stdout:55834574854 2024-03-13T07:34:57.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 10 librados: watch_flush enter 2024-03-13T07:34:57.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 10 librados: watch_flush exit 2024-03-13T07:34:57.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 20 client.4334.objecter shutdown clearing up homeless session... 2024-03-13T07:34:57.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 10 client.4334.objecter successfully canceled tick 2024-03-13T07:34:57.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 1 -- 172.21.15.132:0/2715453845 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f534806d590 msgr2=0x7f534806fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 1 --2- 172.21.15.132:0/2715453845 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f534806d590 0x7f534806fa50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f534c036000 tx=0x7f534c030000 comp rx=0 tx=0).stop 2024-03-13T07:34:57.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 10 monclient: shutdown 2024-03-13T07:34:57.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:57.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 1 -- 172.21.15.132:0/2715453845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f535c13fd40 msgr2=0x7f535c0a9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 1 --2- 172.21.15.132:0/2715453845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f535c13fd40 0x7f535c0a9740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f535400c730 tx=0x7f535400cb30 comp rx=0 tx=0).stop 2024-03-13T07:34:57.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 1 -- 172.21.15.132:0/2715453845 shutdown_connections 2024-03-13T07:34:57.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 1 --2- 172.21.15.132:0/2715453845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f535c13fd40 0x7f535c0a9740 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 1 --2- 172.21.15.132:0/2715453845 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f534806d590 0x7f534806fa50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 1 --2- 172.21.15.132:0/2715453845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f535c0b5320 0x7f535c1d8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 1 --2- 172.21.15.132:0/2715453845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f535c0b1ac0 0x7f535c1d82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 1 -- 172.21.15.132:0/2715453845 >> 172.21.15.132:0/2715453845 conn(0x7f535c06d2b0 msgr2=0x7f535c13f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:57.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 1 -- 172.21.15.132:0/2715453845 shutdown_connections 2024-03-13T07:34:57.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 1 -- 172.21.15.132:0/2715453845 wait complete. 2024-03-13T07:34:57.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.792+0000 7f53625d8640 1 librados: shutdown 2024-03-13T07:34:57.803 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-03-13T07:34:57.803 DEBUG:teuthology.parallel:result is None 2024-03-13T07:34:57.803 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-13T07:34:57.804 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-13T07:34:57.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.903+0000 7f2fc660c640 10 monclient: get_monmap_and_config 2024-03-13T07:34:57.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.903+0000 7f2fc660c640 10 monclient: build_initial_monmap 2024-03-13T07:34:57.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.903+0000 7f2fc660c640 10 monclient: monmap: 2024-03-13T07:34:57.903 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:57.903 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:57.904 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:57.904375+0000 2024-03-13T07:34:57.904 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:57.904375+0000 2024-03-13T07:34:57.904 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:57.904 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.904 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:57.904 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:57.904 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:57.904 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:57.904 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:57.904 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.904+0000 7f2fc660c640 1 Processor -- start 2024-03-13T07:34:57.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.904+0000 7f2fc660c640 1 -- start start 2024-03-13T07:34:57.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.904+0000 7f2fc660c640 10 monclient: init 2024-03-13T07:34:57.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.904+0000 7f2fc660c640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:57.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.904+0000 7f2fc660c640 10 monclient: _add_conns ranks=[3,1,0,2,4] 2024-03-13T07:34:57.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.904+0000 7f2fc660c640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2fc0066c70 0x7f2fc013f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:57.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.904+0000 7f2fc660c640 10 monclient(hunting): picked mon.noname-a con 0x7f2fc0066c70 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:57.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.904+0000 7f2fc660c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2fc013fb20 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:57.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.904+0000 7f2fc660c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2fc01461d0 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:57.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.904+0000 7f2fc660c640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc006fd10 con 0x7f2fc01461d0 2024-03-13T07:34:57.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.905+0000 7f2fc660c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc00b0100 con 0x7f2fc013fb20 2024-03-13T07:34:57.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.905+0000 7f2fc660c640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:57.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.905+0000 7f2fc660c640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc0067050 con 0x7f2fc0066c70 2024-03-13T07:34:57.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.905+0000 7f2fc660c640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:57.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.905+0000 7f2fc660c640 10 monclient(hunting): authenticate will time out at 1325.313106s 2024-03-13T07:34:57.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.905+0000 7f2fbffff640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2fc0066c70 0x7f2fc013f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:57.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.905+0000 7f2fbf7fe640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f2fc013fb20 0x7f2fc0145ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.132:32882/0 (socket says 172.21.15.132:32882) 2024-03-13T07:34:57.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.905+0000 7f2fbf7fe640 1 -- 172.21.15.132:0/870097121 learned_addr learned my addr 172.21.15.132:0/870097121 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T07:34:57.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.906+0000 7f2fbffff640 10 monclient(hunting): get_auth_request con 0x7f2fc0066c70 auth_method 0 2024-03-13T07:34:57.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.906+0000 7f2fbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:57.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.906+0000 7f2fbffff640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:57.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.906+0000 7f2fbffff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:57.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.906+0000 7f2fbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:57.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.906+0000 7f2fbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:57.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.907+0000 7f2fbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:57.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.908+0000 7f2fbeffd640 1 -- 172.21.15.132:0/870097121 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801797002 0 0) 0x7f2fc00b0100 con 0x7f2fc013fb20 2024-03-13T07:34:57.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.908+0000 7f2fbeffd640 10 monclient(hunting): _negotiate 2024-03-13T07:34:57.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.908+0000 7f2fbeffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:57.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.908+0000 7f2fbeffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:57.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.908+0000 7f2fbeffd640 10 monclient(hunting): my global_id is 4351 2024-03-13T07:34:57.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.908+0000 7f2fbeffd640 1 -- 172.21.15.132:0/870097121 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fa8003520 con 0x7f2fc013fb20 2024-03-13T07:34:57.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.908+0000 7f2fbffff640 10 monclient(hunting): handle_auth_done global_id 4341 payload 274 2024-03-13T07:34:57.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.908+0000 7f2fbffff640 1 -- 172.21.15.132:0/870097121 >> v1:172.21.15.195:6789/0 conn(0x7f2fc013fb20 legacy=0x7f2fc0145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.908+0000 7f2fbffff640 1 -- 172.21.15.132:0/870097121 >> v1:172.21.15.132:6789/0 conn(0x7f2fc01461d0 legacy=0x7f2fc014ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.908+0000 7f2fbffff640 10 monclient: _finish_hunting 0 2024-03-13T07:34:57.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.908+0000 7f2fbffff640 1 monclient: found mon.noname-a 2024-03-13T07:34:57.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.908+0000 7f2fbffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:57.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.908+0000 7f2fbffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:57.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.908+0000 7f2fbffff640 1 -- 172.21.15.132:0/870097121 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc00af3c0 con 0x7f2fc0066c70 2024-03-13T07:34:57.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.914+0000 7f2fbffff640 1 --2- 172.21.15.132:0/870097121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc0066c70 0x7f2fc013f5e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00097a0 tx=0x7f2fb002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39026057b1fd9f37 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:57.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fbeffd640 1 -- 172.21.15.132:0/870097121 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0035020 con 0x7f2fc0066c70 2024-03-13T07:34:57.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fbeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:57.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fbeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:57.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fbeffd640 10 monclient: dump: 2024-03-13T07:34:57.924 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:57.924 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:57.924 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.924 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.924 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:57.924 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.924 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:57.924 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:57.924 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:57.924 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fbeffd640 10 monclient: _finish_auth 0 2024-03-13T07:34:57.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fbeffd640 10 monclient: _check_auth_tickets 2024-03-13T07:34:57.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fbeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:57.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fbeffd640 1 -- 172.21.15.132:0/870097121 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb0031830 con 0x7f2fc0066c70 2024-03-13T07:34:57.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fc660c640 5 monclient: authenticate success, global_id 4341 2024-03-13T07:34:57.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fc660c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:34:57.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fbeffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:57.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fc660c640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:57.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fbeffd640 1 -- 172.21.15.132:0/870097121 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0031c00 con 0x7f2fc0066c70 2024-03-13T07:34:57.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fc660c640 4 set_mon_vals no callback set 2024-03-13T07:34:57.926 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fbeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:57.926 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fbeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:57.926 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fbeffd640 10 monclient: dump: 2024-03-13T07:34:57.926 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:57.926 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:57.926 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.926 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.926 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:57.926 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.926 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:57.926 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:57.927 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:57.927 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fc660c640 10 monclient: shutdown 2024-03-13T07:34:57.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fc660c640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:57.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fc660c640 1 -- 172.21.15.132:0/870097121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc0066c70 msgr2=0x7f2fc013f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fc660c640 1 --2- 172.21.15.132:0/870097121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc0066c70 0x7f2fc013f5e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00097a0 tx=0x7f2fb002b8d0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fc660c640 1 -- 172.21.15.132:0/870097121 shutdown_connections 2024-03-13T07:34:57.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fc660c640 1 --2- 172.21.15.132:0/870097121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc0066c70 0x7f2fc013f5e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.924+0000 7f2fc660c640 1 -- 172.21.15.132:0/870097121 >> 172.21.15.132:0/870097121 conn(0x7f2fc006b0e0 msgr2=0x7f2fc006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:57.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.925+0000 7f2fc660c640 1 -- 172.21.15.132:0/870097121 shutdown_connections 2024-03-13T07:34:57.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.925+0000 7f2fc660c640 1 -- 172.21.15.132:0/870097121 wait complete. 2024-03-13T07:34:57.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.925+0000 7f2fc660c640 10 monclient: build_initial_monmap 2024-03-13T07:34:57.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.925+0000 7f2fc660c640 10 monclient: monmap: 2024-03-13T07:34:57.931 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:57.931 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:57.931 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:57.931 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:57.931 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:57.931 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.932 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:57.932 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:57.932 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:57.932 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.925+0000 7f2fc660c640 1 librados: starting msgr at 2024-03-13T07:34:57.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.925+0000 7f2fc660c640 1 librados: starting objecter 2024-03-13T07:34:57.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 1 Processor -- start 2024-03-13T07:34:57.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 1 -- start start 2024-03-13T07:34:57.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 1 librados: setting wanted keys 2024-03-13T07:34:57.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 1 librados: calling monclient init 2024-03-13T07:34:57.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 10 monclient: init 2024-03-13T07:34:57.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:57.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 10 monclient: _add_conns ranks=[0,1,2] 2024-03-13T07:34:57.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc0066c70 0x7f2fc01d7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:57.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 10 monclient(hunting): picked mon.noname-a con 0x7f2fc0066c70 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:57.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2fc013fb20 0x7f2fc011c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:57.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 10 monclient(hunting): picked mon.noname-b con 0x7f2fc013fb20 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:57.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fc01461d0 0x7f2fc011d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:57.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 10 monclient(hunting): picked mon.noname-c con 0x7f2fc01461d0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:57.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:57.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00b0620 con 0x7f2fc0066c70 2024-03-13T07:34:57.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:57.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc006fac0 con 0x7f2fc013fb20 2024-03-13T07:34:57.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:57.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc011cf40 con 0x7f2fc01461d0 2024-03-13T07:34:57.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:57.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.926+0000 7f2fc660c640 10 monclient(hunting): authenticate will time out at 1325.334870s 2024-03-13T07:34:57.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.927+0000 7f2fc4ba9640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fc01461d0 0x7f2fc011d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:57.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.927+0000 7f2fbf7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2fc013fb20 0x7f2fc011c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:57.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.927+0000 7f2fc4ba9640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fc01461d0 0x7f2fc011d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:57888/0 (socket says 172.21.15.132:57888) 2024-03-13T07:34:57.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.927+0000 7f2fc4ba9640 1 -- 172.21.15.132:0/1355720663 learned_addr learned my addr 172.21.15.132:0/1355720663 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:57.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.927+0000 7f2fc4ba9640 10 monclient(hunting): get_auth_request con 0x7f2fc01461d0 auth_method 0 2024-03-13T07:34:57.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.927+0000 7f2fc4ba9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:57.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.927+0000 7f2fc4ba9640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:57.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.927+0000 7f2fc4ba9640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:57.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.927+0000 7f2fbf7fe640 10 monclient(hunting): get_auth_request con 0x7f2fc013fb20 auth_method 0 2024-03-13T07:34:57.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.927+0000 7f2fbf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:57.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.927+0000 7f2fbf7fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:57.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.927+0000 7f2fbf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:57.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.928+0000 7f2fc4ba9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:57.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.928+0000 7f2fc4ba9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:57.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.928+0000 7f2fc4ba9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:57.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.928+0000 7f2fbf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:57.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.928+0000 7f2fbf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:57.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.928+0000 7f2fbf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:57.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.930+0000 7f2fc4ba9640 10 monclient(hunting): handle_auth_done global_id 4337 payload 824 2024-03-13T07:34:57.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.930+0000 7f2fc4ba9640 1 -- 172.21.15.132:0/1355720663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2fc013fb20 msgr2=0x7f2fc011c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.930+0000 7f2fc4ba9640 1 --2- 172.21.15.132:0/1355720663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2fc013fb20 0x7f2fc011c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.930+0000 7f2fc4ba9640 1 -- 172.21.15.132:0/1355720663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc0066c70 msgr2=0x7f2fc01d7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:57.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.930+0000 7f2fc4ba9640 1 --2- 172.21.15.132:0/1355720663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc0066c70 0x7f2fc01d7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:57.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.930+0000 7f2fc4ba9640 10 monclient: _finish_hunting 0 2024-03-13T07:34:57.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.930+0000 7f2fc4ba9640 1 monclient: found mon.noname-c 2024-03-13T07:34:57.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.930+0000 7f2fc4ba9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:57.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.930+0000 7f2fc4ba9640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:34:57.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.930+0000 7f2fc4ba9640 1 -- 172.21.15.132:0/1355720663 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc01412b0 con 0x7f2fc01461d0 2024-03-13T07:34:57.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.933+0000 7f2fc4ba9640 1 --2- 172.21.15.132:0/1355720663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fc01461d0 0x7f2fc011d460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800c6e0 tx=0x7f2fb800cba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:57.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.933+0000 7f2fbcff9640 10 client.?.objecter ms_handle_connect 0x7f2fc01461d0 2024-03-13T07:34:57.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.933+0000 7f2fbcff9640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:57.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fbcff9640 1 -- 172.21.15.132:0/1355720663 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8003b70 con 0x7f2fc01461d0 2024-03-13T07:34:57.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fbcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:57.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fbcff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:34:57.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fbcff9640 10 monclient: dump: 2024-03-13T07:34:57.938 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:57.938 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:57.938 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.939 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.939 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:57.939 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.939 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:57.939 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:57.939 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:57.939 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fbcff9640 10 monclient: _finish_auth 0 2024-03-13T07:34:57.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fbcff9640 10 monclient: _check_auth_tickets 2024-03-13T07:34:57.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fbcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:57.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fbcff9640 1 -- 172.21.15.132:0/1355720663 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb8003d30 con 0x7f2fc01461d0 2024-03-13T07:34:57.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fbcff9640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:57.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fbcff9640 1 -- 172.21.15.132:0/1355720663 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8005310 con 0x7f2fc01461d0 2024-03-13T07:34:57.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fbcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:57.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fbcff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:57.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fbcff9640 10 monclient: dump: 2024-03-13T07:34:57.940 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:57.940 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:57.940 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.940 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:57.940 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:57.940 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:57.940 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:57.940 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:57.940 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:57.940 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:57.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fc660c640 5 monclient: authenticate success, global_id 4337 2024-03-13T07:34:57.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fc660c640 10 monclient: _renew_subs 2024-03-13T07:34:57.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fc660c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:57.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fc660c640 1 -- 172.21.15.132:0/1355720663 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc01d87b0 con 0x7f2fc01461d0 2024-03-13T07:34:57.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fbdffb640 4 set_mon_vals no callback set 2024-03-13T07:34:57.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fc660c640 10 client.4337.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:57.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fc660c640 10 monclient: _renew_subs 2024-03-13T07:34:57.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fc660c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:57.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fc660c640 1 -- 172.21.15.132:0/1355720663 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc011db80 con 0x7f2fc01461d0 2024-03-13T07:34:57.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.937+0000 7f2fc660c640 1 librados: init done 2024-03-13T07:34:57.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.938+0000 7f2fc660c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:57.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.938+0000 7f2fc660c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:57.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.938+0000 7f2fc660c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:34:57.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:57.938+0000 7f2fc660c640 1 -- 172.21.15.132:0/1355720663 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc011d140 con 0x7f2fc01461d0 2024-03-13T07:34:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.208+0000 7f2fbcff9640 1 -- 172.21.15.132:0/1355720663 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2fb80054b0 con 0x7f2fc01461d0 2024-03-13T07:34:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.209+0000 7f2fbcff9640 1 --2- 172.21.15.132:0/1355720663 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2fac06d560 0x7f2fac06fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.209+0000 7f2fbffff640 1 --2- 172.21.15.132:0/1355720663 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2fac06d560 0x7f2fac06fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.209+0000 7f2fbffff640 10 monclient: get_auth_request con 0x7f2fac06d560 auth_method 0 2024-03-13T07:34:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.209+0000 7f2fbffff640 1 --2- 172.21.15.132:0/1355720663 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2fac06d560 0x7f2fac06fa20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0036000 tx=0x7f2fb0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.209+0000 7f2fbcff9640 10 client.4337.objecter ms_handle_connect 0x7f2fac06d560 2024-03-13T07:34:58.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.224+0000 7f2fbcff9640 1 -- 172.21.15.132:0/1355720663 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2fb80057a0 con 0x7f2fc01461d0 2024-03-13T07:34:58.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.224+0000 7f2fbcff9640 10 client.4337.objecter ms_dispatch 0x7f2fc0000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:58.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.224+0000 7f2fbcff9640 3 client.4337.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:58.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.224+0000 7f2fbcff9640 3 client.4337.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:58.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.225+0000 7f2fbcff9640 20 client.4337.objecter dump_active .. 0 homeless 2024-03-13T07:34:58.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.225+0000 7f2fbcff9640 1 -- 172.21.15.132:0/1355720663 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f2fc011d140 con 0x7f2fc01461d0 2024-03-13T07:34:58.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.225+0000 7f2fbcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:58.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.225+0000 7f2fbcff9640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:58.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.415+0000 7f2fc660c640 1 -- 172.21.15.132:0/1355720663 --> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2fc011ef60 con 0x7f2fac06d560 2024-03-13T07:34:58.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.416+0000 7f2fbcff9640 1 -- 172.21.15.132:0/1355720663 <== mgr.4104 v2:172.21.15.132:6800/3847550149 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+35521 (secure 0 0 0) 0x7f2fc011ef60 con 0x7f2fac06d560 2024-03-13T07:34:58.416 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:34:58.417 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-03-13T07:34:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 10 librados: watch_flush enter 2024-03-13T07:34:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 10 librados: watch_flush exit 2024-03-13T07:34:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 20 client.4337.objecter shutdown clearing up homeless session... 2024-03-13T07:34:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 10 client.4337.objecter successfully canceled tick 2024-03-13T07:34:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 1 -- 172.21.15.132:0/1355720663 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2fac06d560 msgr2=0x7f2fac06fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 1 --2- 172.21.15.132:0/1355720663 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2fac06d560 0x7f2fac06fa20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0036000 tx=0x7f2fb0030000 comp rx=0 tx=0).stop 2024-03-13T07:34:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 10 monclient: shutdown 2024-03-13T07:34:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 1 -- 172.21.15.132:0/1355720663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fc01461d0 msgr2=0x7f2fc011d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 1 --2- 172.21.15.132:0/1355720663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fc01461d0 0x7f2fc011d460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800c6e0 tx=0x7f2fb800cba0 comp rx=0 tx=0).stop 2024-03-13T07:34:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 1 -- 172.21.15.132:0/1355720663 shutdown_connections 2024-03-13T07:34:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 1 --2- 172.21.15.132:0/1355720663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fc01461d0 0x7f2fc011d460 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 1 --2- 172.21.15.132:0/1355720663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2fc013fb20 0x7f2fc011c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 1 --2- 172.21.15.132:0/1355720663 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2fac06d560 0x7f2fac06fa20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 1 --2- 172.21.15.132:0/1355720663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc0066c70 0x7f2fc01d7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 1 -- 172.21.15.132:0/1355720663 >> 172.21.15.132:0/1355720663 conn(0x7f2fc006b0e0 msgr2=0x7f2fc0070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:58.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 1 -- 172.21.15.132:0/1355720663 shutdown_connections 2024-03-13T07:34:58.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 1 -- 172.21.15.132:0/1355720663 wait complete. 2024-03-13T07:34:58.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.418+0000 7f2fc660c640 1 librados: shutdown 2024-03-13T07:34:58.428 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-03-13T07:34:57.067694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218808,"kb_used_data":2640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755912,"statfs":{"total":773094113280,"available":772870053888,"internally_reserved":0,"allocated":2703360,"data_stored":1474828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001023"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-13T07:34:54.833668+0000","last_change":"2024-03-13T07:34:52.389651+0000","last_active":"2024-03-13T07:34:54.833668+0000","last_peered":"2024-03-13T07:34:54.833668+0000","last_clean":"2024-03-13T07:34:54.833668+0000","last_became_active":"2024-03-13T07:34:52.384430+0000","last_became_peered":"2024-03-13T07:34:52.384430+0000","last_unstale":"2024-03-13T07:34:54.833668+0000","last_undegraded":"2024-03-13T07:34:54.833668+0000","last_fullsized":"2024-03-13T07:34:54.833668+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:19:31.011822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-13T07:34:52.485109+0000","last_change":"2024-03-13T07:34:52.429709+0000","last_active":"2024-03-13T07:34:52.485109+0000","last_peered":"2024-03-13T07:34:52.485109+0000","last_clean":"2024-03-13T07:34:52.485109+0000","last_became_active":"2024-03-13T07:34:52.404260+0000","last_became_peered":"2024-03-13T07:34:52.404260+0000","last_unstale":"2024-03-13T07:34:52.485109+0000","last_undegraded":"2024-03-13T07:34:52.485109+0000","last_fullsized":"2024-03-13T07:34:52.485109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T07:40:34.077043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-13T07:34:54.828266+0000","last_change":"2024-03-13T07:34:52.133418+0000","last_active":"2024-03-13T07:34:54.828266+0000","last_peered":"2024-03-13T07:34:54.828266+0000","last_clean":"2024-03-13T07:34:54.828266+0000","last_became_active":"2024-03-13T07:34:52.128201+0000","last_became_peered":"2024-03-13T07:34:52.128201+0000","last_unstale":"2024-03-13T07:34:54.828266+0000","last_undegraded":"2024-03-13T07:34:54.828266+0000","last_fullsized":"2024-03-13T07:34:54.828266+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T08:41:28.651710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-13T07:34:53.216755+0000","last_change":"2024-03-13T07:34:52.930585+0000","last_active":"2024-03-13T07:34:53.216755+0000","last_peered":"2024-03-13T07:34:53.216755+0000","last_clean":"2024-03-13T07:34:53.216755+0000","last_became_active":"2024-03-13T07:34:52.922427+0000","last_became_peered":"2024-03-13T07:34:52.922427+0000","last_unstale":"2024-03-13T07:34:53.216755+0000","last_undegraded":"2024-03-13T07:34:53.216755+0000","last_fullsized":"2024-03-13T07:34:53.216755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:20:14.141648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-13T07:34:54.820377+0000","last_change":"2024-03-13T07:34:52.914984+0000","last_active":"2024-03-13T07:34:54.820377+0000","last_peered":"2024-03-13T07:34:54.820377+0000","last_clean":"2024-03-13T07:34:54.820377+0000","last_became_active":"2024-03-13T07:34:52.909827+0000","last_became_peered":"2024-03-13T07:34:52.909827+0000","last_unstale":"2024-03-13T07:34:54.820377+0000","last_undegraded":"2024-03-13T07:34:54.820377+0000","last_fullsized":"2024-03-13T07:34:54.820377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T10:06:44.279005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-13T07:34:54.026700+0000","last_change":"2024-03-13T07:34:52.044522+0000","last_active":"2024-03-13T07:34:54.026700+0000","last_peered":"2024-03-13T07:34:54.026700+0000","last_clean":"2024-03-13T07:34:54.026700+0000","last_became_active":"2024-03-13T07:34:52.014922+0000","last_became_peered":"2024-03-13T07:34:52.014922+0000","last_unstale":"2024-03-13T07:34:54.026700+0000","last_undegraded":"2024-03-13T07:34:54.026700+0000","last_fullsized":"2024-03-13T07:34:54.026700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T16:43:03.627625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-13T07:34:54.806535+0000","last_change":"2024-03-13T07:34:52.050005+0000","last_active":"2024-03-13T07:34:54.806535+0000","last_peered":"2024-03-13T07:34:54.806535+0000","last_clean":"2024-03-13T07:34:54.806535+0000","last_became_active":"2024-03-13T07:34:52.021695+0000","last_became_peered":"2024-03-13T07:34:52.021695+0000","last_unstale":"2024-03-13T07:34:54.806535+0000","last_undegraded":"2024-03-13T07:34:54.806535+0000","last_fullsized":"2024-03-13T07:34:54.806535+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:11:09.006656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-13T07:34:54.825351+0000","last_change":"2024-03-13T07:34:52.336528+0000","last_active":"2024-03-13T07:34:54.825351+0000","last_peered":"2024-03-13T07:34:54.825351+0000","last_clean":"2024-03-13T07:34:54.825351+0000","last_became_active":"2024-03-13T07:34:52.331245+0000","last_became_peered":"2024-03-13T07:34:52.331245+0000","last_unstale":"2024-03-13T07:34:54.825351+0000","last_undegraded":"2024-03-13T07:34:54.825351+0000","last_fullsized":"2024-03-13T07:34:54.825351+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:06:57.704465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'76","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-13T07:34:53.189195+0000","last_change":"2024-03-13T07:34:48.043475+0000","last_active":"2024-03-13T07:34:53.189195+0000","last_peered":"2024-03-13T07:34:53.189195+0000","last_clean":"2024-03-13T07:34:53.189195+0000","last_became_active":"2024-03-13T07:34:47.400882+0000","last_became_peered":"2024-03-13T07:34:47.400882+0000","last_unstale":"2024-03-13T07:34:53.189195+0000","last_undegraded":"2024-03-13T07:34:53.189195+0000","last_fullsized":"2024-03-13T07:34:53.189195+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:34.136507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:34.136507+0000","last_clean_scrub_stamp":"2024-03-13T07:34:34.136507+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:24:34.761011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":20,"seq":85899345928,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":84809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":20,"seq":85899345929,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":79399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":20,"seq":85899345930,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":638976,"data_stored":482649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":18,"seq":77309411338,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":79399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476743,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":90557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":90557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":84809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":638976,"data_stored":482649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-13T07:34:58.430 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-13T07:34:58.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.530+0000 7ff63be47640 10 monclient: get_monmap_and_config 2024-03-13T07:34:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.530+0000 7ff63be47640 10 monclient: build_initial_monmap 2024-03-13T07:34:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.530+0000 7ff63be47640 10 monclient: monmap: 2024-03-13T07:34:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:58.531059+0000 2024-03-13T07:34:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:58.531059+0000 2024-03-13T07:34:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:58.531 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:58.531 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:58.531 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:58.531 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:58.531 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:58.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.530+0000 7ff63be47640 1 Processor -- start 2024-03-13T07:34:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.531+0000 7ff63be47640 1 -- start start 2024-03-13T07:34:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.531+0000 7ff63be47640 10 monclient: init 2024-03-13T07:34:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.531+0000 7ff63be47640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.531+0000 7ff63be47640 10 monclient: _add_conns ranks=[3,4,1,0,2] 2024-03-13T07:34:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.531+0000 7ff63be47640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff6340b5320 0x7ff6340b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.531+0000 7ff63be47640 10 monclient(hunting): picked mon.noname-a con 0x7ff6340b5320 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.531+0000 7ff63be47640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff6340b1ac0 0x7ff6340b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.531+0000 7ff63be47640 10 monclient(hunting): picked mon.noname-c con 0x7ff6340b1ac0 addr v2:172.21.15.195:3300/0 2024-03-13T07:34:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.531+0000 7ff63be47640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff63413f630 addr v1:172.21.15.195:6789/0 2024-03-13T07:34:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.531+0000 7ff63be47640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6340b1150 con 0x7ff63413f630 2024-03-13T07:34:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.531+0000 7ff63be47640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.531+0000 7ff63be47640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6340b13c0 con 0x7ff6340b5320 2024-03-13T07:34:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.531+0000 7ff63be47640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.531+0000 7ff63be47640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6340b16a0 con 0x7ff6340b1ac0 2024-03-13T07:34:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.531+0000 7ff63be47640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:58.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.531+0000 7ff63be47640 10 monclient(hunting): authenticate will time out at 1325.939758s 2024-03-13T07:34:58.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.532+0000 7ff6393e2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff6340b1ac0 0x7ff6340b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:58.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.532+0000 7ff639be3640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff6340b5320 0x7ff6340b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:58.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.532+0000 7ff6393e2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff6340b1ac0 0x7ff6340b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:56158/0 (socket says 172.21.15.132:56158) 2024-03-13T07:34:58.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.532+0000 7ff6393e2640 1 -- 172.21.15.132:0/3718803177 learned_addr learned my addr 172.21.15.132:0/3718803177 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:58.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.532+0000 7ff6393e2640 10 monclient(hunting): get_auth_request con 0x7ff6340b1ac0 auth_method 0 2024-03-13T07:34:58.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.532+0000 7ff6393e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:58.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.532+0000 7ff6393e2640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:58.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.532+0000 7ff6393e2640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:58.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.532+0000 7ff639be3640 10 monclient(hunting): get_auth_request con 0x7ff6340b5320 auth_method 0 2024-03-13T07:34:58.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.532+0000 7ff639be3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:58.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.532+0000 7ff639be3640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:58.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.532+0000 7ff639be3640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:58.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.533+0000 7ff639be3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:58.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.533+0000 7ff639be3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:58.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.533+0000 7ff639be3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:58.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.533+0000 7ff6393e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:58.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.533+0000 7ff6393e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:58.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.533+0000 7ff6393e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:58.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.534+0000 7ff639be3640 10 monclient(hunting): handle_auth_done global_id 4347 payload 274 2024-03-13T07:34:58.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.534+0000 7ff639be3640 1 -- 172.21.15.132:0/3718803177 >> v2:172.21.15.195:3300/0 conn(0x7ff6340b1ac0 msgr2=0x7ff6340b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:58.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.534+0000 7ff639be3640 1 --2- 172.21.15.132:0/3718803177 >> v2:172.21.15.195:3300/0 conn(0x7ff6340b1ac0 0x7ff6340b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:58.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.534+0000 7ff639be3640 1 -- 172.21.15.132:0/3718803177 >> v1:172.21.15.195:6789/0 conn(0x7ff63413f630 legacy=0x7ff63413fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:58.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.534+0000 7ff639be3640 10 monclient: _finish_hunting 0 2024-03-13T07:34:58.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.534+0000 7ff639be3640 1 monclient: found mon.noname-a 2024-03-13T07:34:58.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.534+0000 7ff639be3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:58.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.534+0000 7ff639be3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:58.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.534+0000 7ff639be3640 1 -- 172.21.15.132:0/3718803177 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6340afa20 con 0x7ff6340b5320 2024-03-13T07:34:58.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.536+0000 7ff639be3640 1 --2- 172.21.15.132:0/3718803177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6340b5320 0x7ff6340b0220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff63406ddf0 tx=0x7ff62402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad566ccb54b5de8 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:58.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.540+0000 7ff638be1640 1 -- 172.21.15.132:0/3718803177 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624038070 con 0x7ff6340b5320 2024-03-13T07:34:58.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.540+0000 7ff638be1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:58.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.540+0000 7ff638be1640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:58.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.540+0000 7ff638be1640 10 monclient: dump: 2024-03-13T07:34:58.540 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:58.540 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:58.540 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:58.541 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:58.541 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:58.541 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:58.541 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:58.541 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:58.541 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:58.541 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:58.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.540+0000 7ff638be1640 10 monclient: _finish_auth 0 2024-03-13T07:34:58.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.540+0000 7ff638be1640 10 monclient: _check_auth_tickets 2024-03-13T07:34:58.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.540+0000 7ff638be1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:58.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.540+0000 7ff638be1640 1 -- 172.21.15.132:0/3718803177 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff624031710 con 0x7ff6340b5320 2024-03-13T07:34:58.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.540+0000 7ff638be1640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:58.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.540+0000 7ff638be1640 1 -- 172.21.15.132:0/3718803177 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624031ae0 con 0x7ff6340b5320 2024-03-13T07:34:58.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.540+0000 7ff638be1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:58.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.540+0000 7ff638be1640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:58.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.540+0000 7ff638be1640 10 monclient: dump: 2024-03-13T07:34:58.542 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:58.542 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:58.542 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:58.542 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:58.542 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:58.543 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:58.543 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:58.543 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:58.543 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:58.543 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:58.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.540+0000 7ff63be47640 5 monclient: authenticate success, global_id 4347 2024-03-13T07:34:58.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.540+0000 7ff63be47640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:58.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.540+0000 7ff63be47640 4 set_mon_vals no callback set 2024-03-13T07:34:58.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.541+0000 7ff63be47640 10 monclient: shutdown 2024-03-13T07:34:58.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.541+0000 7ff63be47640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:58.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.541+0000 7ff63be47640 1 -- 172.21.15.132:0/3718803177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6340b5320 msgr2=0x7ff6340b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:58.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.541+0000 7ff63be47640 1 --2- 172.21.15.132:0/3718803177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6340b5320 0x7ff6340b0220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff63406ddf0 tx=0x7ff62402b7b0 comp rx=0 tx=0).stop 2024-03-13T07:34:58.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.541+0000 7ff63be47640 1 -- 172.21.15.132:0/3718803177 shutdown_connections 2024-03-13T07:34:58.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.541+0000 7ff63be47640 1 --2- 172.21.15.132:0/3718803177 >> v2:172.21.15.195:3300/0 conn(0x7ff6340b1ac0 0x7ff6340b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:58.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.541+0000 7ff63be47640 1 --2- 172.21.15.132:0/3718803177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6340b5320 0x7ff6340b0220 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:58.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.541+0000 7ff63be47640 1 -- 172.21.15.132:0/3718803177 >> 172.21.15.132:0/3718803177 conn(0x7ff63406d2b0 msgr2=0x7ff63406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:58.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.541+0000 7ff63be47640 1 -- 172.21.15.132:0/3718803177 shutdown_connections 2024-03-13T07:34:58.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.541+0000 7ff63be47640 1 -- 172.21.15.132:0/3718803177 wait complete. 2024-03-13T07:34:58.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.542+0000 7ff63be47640 10 monclient: build_initial_monmap 2024-03-13T07:34:58.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.542+0000 7ff63be47640 10 monclient: monmap: 2024-03-13T07:34:58.549 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:58.549 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:58.549 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:58.549 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:58.549 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:58.549 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:58.549 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:58.549 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:58.549 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:58.549 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:58.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.542+0000 7ff63be47640 1 librados: starting msgr at 2024-03-13T07:34:58.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.542+0000 7ff63be47640 1 librados: starting objecter 2024-03-13T07:34:58.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.542+0000 7ff63be47640 1 Processor -- start 2024-03-13T07:34:58.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.542+0000 7ff63be47640 1 -- start start 2024-03-13T07:34:58.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.542+0000 7ff63be47640 1 librados: setting wanted keys 2024-03-13T07:34:58.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.542+0000 7ff63be47640 1 librados: calling monclient init 2024-03-13T07:34:58.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.542+0000 7ff63be47640 10 monclient: init 2024-03-13T07:34:58.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63be47640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:58.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63be47640 10 monclient: _add_conns ranks=[2,0,1] 2024-03-13T07:34:58.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63be47640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6340b1ac0 0x7ff634142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:58.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63be47640 10 monclient(hunting): picked mon.noname-c con 0x7ff6340b1ac0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:58.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63be47640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6340b5320 0x7ff63413b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:58.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63be47640 10 monclient(hunting): picked mon.noname-a con 0x7ff6340b5320 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:58.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63be47640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff63413f630 0x7ff634142b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:58.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63be47640 10 monclient(hunting): picked mon.noname-b con 0x7ff63413f630 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:58.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63be47640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:58.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63be47640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6340b4980 con 0x7ff6340b5320 2024-03-13T07:34:58.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63be47640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:58.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63be47640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6340b4800 con 0x7ff63413f630 2024-03-13T07:34:58.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63be47640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:58.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63be47640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63414cef0 con 0x7ff6340b1ac0 2024-03-13T07:34:58.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63be47640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:58.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63be47640 10 monclient(hunting): authenticate will time out at 1325.951186s 2024-03-13T07:34:58.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff6393e2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6340b5320 0x7ff63413b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:58.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff639be3640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6340b1ac0 0x7ff634142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:58.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff6393e2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6340b5320 0x7ff63413b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34684/0 (socket says 172.21.15.132:34684) 2024-03-13T07:34:58.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff6393e2640 1 -- 172.21.15.132:0/1534542187 learned_addr learned my addr 172.21.15.132:0/1534542187 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:58.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff6393e2640 10 monclient(hunting): get_auth_request con 0x7ff6340b5320 auth_method 0 2024-03-13T07:34:58.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff6393e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:58.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff6393e2640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:58.551 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff6393e2640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:58.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.543+0000 7ff63a3e4640 1 --2- 172.21.15.132:0/1534542187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff63413f630 0x7ff634142b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:58.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff639be3640 10 monclient(hunting): get_auth_request con 0x7ff6340b1ac0 auth_method 0 2024-03-13T07:34:58.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff639be3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:58.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff639be3640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:58.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff639be3640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:58.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff63a3e4640 10 monclient(hunting): get_auth_request con 0x7ff63413f630 auth_method 0 2024-03-13T07:34:58.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff63a3e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:58.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff63a3e4640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:58.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff63a3e4640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:58.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff6393e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:58.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff6393e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:58.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff6393e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:58.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff63a3e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:58.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff63a3e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:58.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff63a3e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:58.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff639be3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:58.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff639be3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:58.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.544+0000 7ff639be3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:58.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.547+0000 7ff6393e2640 10 monclient(hunting): handle_auth_done global_id 4350 payload 824 2024-03-13T07:34:58.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.547+0000 7ff6393e2640 1 -- 172.21.15.132:0/1534542187 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6340b1ac0 msgr2=0x7ff634142110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:58.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.547+0000 7ff6393e2640 1 --2- 172.21.15.132:0/1534542187 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6340b1ac0 0x7ff634142110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:58.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.547+0000 7ff6393e2640 1 -- 172.21.15.132:0/1534542187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff63413f630 msgr2=0x7ff634142b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:58.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.547+0000 7ff6393e2640 1 --2- 172.21.15.132:0/1534542187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff63413f630 0x7ff634142b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:58.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.547+0000 7ff6393e2640 10 monclient: _finish_hunting 0 2024-03-13T07:34:58.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.547+0000 7ff6393e2640 1 monclient: found mon.noname-a 2024-03-13T07:34:58.554 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.547+0000 7ff6393e2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:58.554 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.547+0000 7ff6393e2640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T07:34:58.554 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.547+0000 7ff6393e2640 1 -- 172.21.15.132:0/1534542187 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63413b700 con 0x7ff6340b5320 2024-03-13T07:34:58.554 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.547+0000 7ff63a3e4640 1 --2- 172.21.15.132:0/1534542187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff63413f630 0x7ff634142b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-13T07:34:58.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.549+0000 7ff6393e2640 1 --2- 172.21.15.132:0/1534542187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6340b5320 0x7ff63413b1c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff630009fd0 tx=0x7ff63000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:58.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.549+0000 7ff62a7fc640 10 client.?.objecter ms_handle_connect 0x7ff6340b5320 2024-03-13T07:34:58.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.549+0000 7ff62a7fc640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:58.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff62a7fc640 1 -- 172.21.15.132:0/1534542187 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630017070 con 0x7ff6340b5320 2024-03-13T07:34:58.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff62a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:58.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff62a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T07:34:58.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff62a7fc640 10 monclient: dump: 2024-03-13T07:34:58.556 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:58.556 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:58.556 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:58.556 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:58.557 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:58.557 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:58.557 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:58.557 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:58.557 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:58.557 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:58.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff62a7fc640 10 monclient: _finish_auth 0 2024-03-13T07:34:58.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff62a7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:34:58.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff62a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:58.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff62a7fc640 1 -- 172.21.15.132:0/1534542187 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63000c940 con 0x7ff6340b5320 2024-03-13T07:34:58.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff63be47640 5 monclient: authenticate success, global_id 4350 2024-03-13T07:34:58.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff62a7fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:58.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff62a7fc640 1 -- 172.21.15.132:0/1534542187 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63000cd10 con 0x7ff6340b5320 2024-03-13T07:34:58.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff63be47640 10 monclient: _renew_subs 2024-03-13T07:34:58.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff63be47640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:58.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff63be47640 1 -- 172.21.15.132:0/1534542187 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6340aa180 con 0x7ff6340b5320 2024-03-13T07:34:58.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff62a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:58.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff62a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T07:34:58.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff62bfff640 4 set_mon_vals no callback set 2024-03-13T07:34:58.558 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff62a7fc640 10 monclient: dump: 2024-03-13T07:34:58.558 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:58.558 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:58.558 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:58.558 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:58.558 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:58.558 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:58.558 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:58.558 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:58.558 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:58.558 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:58.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff63be47640 10 client.4350.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:58.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff63be47640 10 monclient: _renew_subs 2024-03-13T07:34:58.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff63be47640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:58.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff63be47640 1 -- 172.21.15.132:0/1534542187 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff634143280 con 0x7ff6340b5320 2024-03-13T07:34:58.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.553+0000 7ff63be47640 1 librados: init done 2024-03-13T07:34:58.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.554+0000 7ff63be47640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:58.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.554+0000 7ff63be47640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:58.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.554+0000 7ff63be47640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T07:34:58.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.554+0000 7ff63be47640 1 -- 172.21.15.132:0/1534542187 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6340aa350 con 0x7ff6340b5320 2024-03-13T07:34:58.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.573+0000 7ff62a7fc640 1 -- 172.21.15.132:0/1534542187 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7ff63001a070 con 0x7ff6340b5320 2024-03-13T07:34:58.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.573+0000 7ff62a7fc640 1 --2- 172.21.15.132:0/1534542187 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7ff61806d560 0x7ff61806fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:58.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.573+0000 7ff639be3640 1 --2- 172.21.15.132:0/1534542187 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7ff61806d560 0x7ff61806fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:58.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.573+0000 7ff639be3640 10 monclient: get_auth_request con 0x7ff61806d560 auth_method 0 2024-03-13T07:34:58.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.574+0000 7ff639be3640 1 --2- 172.21.15.132:0/1534542187 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7ff61806d560 0x7ff61806fa20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff624035000 tx=0x7ff624008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:58.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.574+0000 7ff62a7fc640 10 client.4350.objecter ms_handle_connect 0x7ff61806d560 2024-03-13T07:34:58.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.575+0000 7ff62a7fc640 1 -- 172.21.15.132:0/1534542187 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff630057690 con 0x7ff6340b5320 2024-03-13T07:34:58.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.575+0000 7ff62a7fc640 10 client.4350.objecter ms_dispatch 0x7ff634000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:58.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.575+0000 7ff62a7fc640 3 client.4350.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:58.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.575+0000 7ff62a7fc640 3 client.4350.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:58.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.576+0000 7ff62a7fc640 20 client.4350.objecter dump_active .. 0 homeless 2024-03-13T07:34:58.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.808+0000 7ff62a7fc640 1 -- 172.21.15.132:0/1534542187 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7ff6300595f0 con 0x7ff6340b5320 2024-03-13T07:34:58.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.808+0000 7ff62a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:58.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.808+0000 7ff62a7fc640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:58.975 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.975+0000 7ff63be47640 1 -- 172.21.15.132:0/1534542187 --> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff63404f000 con 0x7ff61806d560 2024-03-13T07:34:58.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.976+0000 7ff62a7fc640 1 -- 172.21.15.132:0/1534542187 <== mgr.4104 v2:172.21.15.132:6800/3847550149 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+35521 (secure 0 0 0) 0x7ff63404f000 con 0x7ff61806d560 2024-03-13T07:34:58.976 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:34:58.977 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-03-13T07:34:58.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.978+0000 7ff63be47640 10 librados: watch_flush enter 2024-03-13T07:34:58.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.978+0000 7ff63be47640 10 librados: watch_flush exit 2024-03-13T07:34:58.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.978+0000 7ff63be47640 20 client.4350.objecter shutdown clearing up homeless session... 2024-03-13T07:34:58.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.978+0000 7ff63be47640 10 client.4350.objecter successfully canceled tick 2024-03-13T07:34:58.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.978+0000 7ff63be47640 1 -- 172.21.15.132:0/1534542187 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7ff61806d560 msgr2=0x7ff61806fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:58.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.978+0000 7ff63be47640 1 --2- 172.21.15.132:0/1534542187 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7ff61806d560 0x7ff61806fa20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff624035000 tx=0x7ff624008030 comp rx=0 tx=0).stop 2024-03-13T07:34:58.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.978+0000 7ff63be47640 10 monclient: shutdown 2024-03-13T07:34:58.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.978+0000 7ff63be47640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:58.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.978+0000 7ff63be47640 1 -- 172.21.15.132:0/1534542187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6340b5320 msgr2=0x7ff63413b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:58.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.978+0000 7ff63be47640 1 --2- 172.21.15.132:0/1534542187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6340b5320 0x7ff63413b1c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff630009fd0 tx=0x7ff63000ed40 comp rx=0 tx=0).stop 2024-03-13T07:34:58.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.978+0000 7ff63be47640 1 -- 172.21.15.132:0/1534542187 shutdown_connections 2024-03-13T07:34:58.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.978+0000 7ff63be47640 1 --2- 172.21.15.132:0/1534542187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff63413f630 0x7ff634142b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:58.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.978+0000 7ff63be47640 1 --2- 172.21.15.132:0/1534542187 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7ff61806d560 0x7ff61806fa20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:58.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.978+0000 7ff63be47640 1 --2- 172.21.15.132:0/1534542187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6340b5320 0x7ff63413b1c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:58.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.978+0000 7ff63be47640 1 --2- 172.21.15.132:0/1534542187 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6340b1ac0 0x7ff634142110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:58.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.978+0000 7ff63be47640 1 -- 172.21.15.132:0/1534542187 >> 172.21.15.132:0/1534542187 conn(0x7ff63406d2b0 msgr2=0x7ff63413d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:58.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.979+0000 7ff63be47640 1 -- 172.21.15.132:0/1534542187 shutdown_connections 2024-03-13T07:34:58.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.979+0000 7ff63be47640 1 -- 172.21.15.132:0/1534542187 wait complete. 2024-03-13T07:34:58.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:58.979+0000 7ff63be47640 1 librados: shutdown 2024-03-13T07:34:58.989 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-03-13T07:34:57.067694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218808,"kb_used_data":2640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755912,"statfs":{"total":773094113280,"available":772870053888,"internally_reserved":0,"allocated":2703360,"data_stored":1474828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001023"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-13T07:34:54.833668+0000","last_change":"2024-03-13T07:34:52.389651+0000","last_active":"2024-03-13T07:34:54.833668+0000","last_peered":"2024-03-13T07:34:54.833668+0000","last_clean":"2024-03-13T07:34:54.833668+0000","last_became_active":"2024-03-13T07:34:52.384430+0000","last_became_peered":"2024-03-13T07:34:52.384430+0000","last_unstale":"2024-03-13T07:34:54.833668+0000","last_undegraded":"2024-03-13T07:34:54.833668+0000","last_fullsized":"2024-03-13T07:34:54.833668+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:19:31.011822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-13T07:34:52.485109+0000","last_change":"2024-03-13T07:34:52.429709+0000","last_active":"2024-03-13T07:34:52.485109+0000","last_peered":"2024-03-13T07:34:52.485109+0000","last_clean":"2024-03-13T07:34:52.485109+0000","last_became_active":"2024-03-13T07:34:52.404260+0000","last_became_peered":"2024-03-13T07:34:52.404260+0000","last_unstale":"2024-03-13T07:34:52.485109+0000","last_undegraded":"2024-03-13T07:34:52.485109+0000","last_fullsized":"2024-03-13T07:34:52.485109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T07:40:34.077043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-13T07:34:54.828266+0000","last_change":"2024-03-13T07:34:52.133418+0000","last_active":"2024-03-13T07:34:54.828266+0000","last_peered":"2024-03-13T07:34:54.828266+0000","last_clean":"2024-03-13T07:34:54.828266+0000","last_became_active":"2024-03-13T07:34:52.128201+0000","last_became_peered":"2024-03-13T07:34:52.128201+0000","last_unstale":"2024-03-13T07:34:54.828266+0000","last_undegraded":"2024-03-13T07:34:54.828266+0000","last_fullsized":"2024-03-13T07:34:54.828266+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T08:41:28.651710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-13T07:34:53.216755+0000","last_change":"2024-03-13T07:34:52.930585+0000","last_active":"2024-03-13T07:34:53.216755+0000","last_peered":"2024-03-13T07:34:53.216755+0000","last_clean":"2024-03-13T07:34:53.216755+0000","last_became_active":"2024-03-13T07:34:52.922427+0000","last_became_peered":"2024-03-13T07:34:52.922427+0000","last_unstale":"2024-03-13T07:34:53.216755+0000","last_undegraded":"2024-03-13T07:34:53.216755+0000","last_fullsized":"2024-03-13T07:34:53.216755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:20:14.141648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-13T07:34:54.820377+0000","last_change":"2024-03-13T07:34:52.914984+0000","last_active":"2024-03-13T07:34:54.820377+0000","last_peered":"2024-03-13T07:34:54.820377+0000","last_clean":"2024-03-13T07:34:54.820377+0000","last_became_active":"2024-03-13T07:34:52.909827+0000","last_became_peered":"2024-03-13T07:34:52.909827+0000","last_unstale":"2024-03-13T07:34:54.820377+0000","last_undegraded":"2024-03-13T07:34:54.820377+0000","last_fullsized":"2024-03-13T07:34:54.820377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T10:06:44.279005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-13T07:34:54.026700+0000","last_change":"2024-03-13T07:34:52.044522+0000","last_active":"2024-03-13T07:34:54.026700+0000","last_peered":"2024-03-13T07:34:54.026700+0000","last_clean":"2024-03-13T07:34:54.026700+0000","last_became_active":"2024-03-13T07:34:52.014922+0000","last_became_peered":"2024-03-13T07:34:52.014922+0000","last_unstale":"2024-03-13T07:34:54.026700+0000","last_undegraded":"2024-03-13T07:34:54.026700+0000","last_fullsized":"2024-03-13T07:34:54.026700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T16:43:03.627625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-13T07:34:54.806535+0000","last_change":"2024-03-13T07:34:52.050005+0000","last_active":"2024-03-13T07:34:54.806535+0000","last_peered":"2024-03-13T07:34:54.806535+0000","last_clean":"2024-03-13T07:34:54.806535+0000","last_became_active":"2024-03-13T07:34:52.021695+0000","last_became_peered":"2024-03-13T07:34:52.021695+0000","last_unstale":"2024-03-13T07:34:54.806535+0000","last_undegraded":"2024-03-13T07:34:54.806535+0000","last_fullsized":"2024-03-13T07:34:54.806535+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:11:09.006656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-13T07:34:54.825351+0000","last_change":"2024-03-13T07:34:52.336528+0000","last_active":"2024-03-13T07:34:54.825351+0000","last_peered":"2024-03-13T07:34:54.825351+0000","last_clean":"2024-03-13T07:34:54.825351+0000","last_became_active":"2024-03-13T07:34:52.331245+0000","last_became_peered":"2024-03-13T07:34:52.331245+0000","last_unstale":"2024-03-13T07:34:54.825351+0000","last_undegraded":"2024-03-13T07:34:54.825351+0000","last_fullsized":"2024-03-13T07:34:54.825351+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:06:57.704465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'76","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-13T07:34:53.189195+0000","last_change":"2024-03-13T07:34:48.043475+0000","last_active":"2024-03-13T07:34:53.189195+0000","last_peered":"2024-03-13T07:34:53.189195+0000","last_clean":"2024-03-13T07:34:53.189195+0000","last_became_active":"2024-03-13T07:34:47.400882+0000","last_became_peered":"2024-03-13T07:34:47.400882+0000","last_unstale":"2024-03-13T07:34:53.189195+0000","last_undegraded":"2024-03-13T07:34:53.189195+0000","last_fullsized":"2024-03-13T07:34:53.189195+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:34.136507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:34.136507+0000","last_clean_scrub_stamp":"2024-03-13T07:34:34.136507+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:24:34.761011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":20,"seq":85899345928,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":84809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":20,"seq":85899345929,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":79399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":20,"seq":85899345930,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":638976,"data_stored":482649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":18,"seq":77309411338,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":79399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476743,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":90557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":90557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":84809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":638976,"data_stored":482649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-13T07:34:58.990 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-13T07:34:58.990 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-13T07:34:58.990 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-13T07:34:58.990 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-13T07:34:59.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.091+0000 7fc6b72d6640 10 monclient: get_monmap_and_config 2024-03-13T07:34:59.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.091+0000 7fc6b72d6640 10 monclient: build_initial_monmap 2024-03-13T07:34:59.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.091+0000 7fc6b72d6640 10 monclient: monmap: 2024-03-13T07:34:59.091 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:59.091 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:59.091 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:34:59.092162+0000 2024-03-13T07:34:59.091 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:34:59.092162+0000 2024-03-13T07:34:59.091 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:59.091 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:59.092 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T07:34:59.092 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T07:34:59.092 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T07:34:59.092 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T07:34:59.092 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T07:34:59.092 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:59.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.091+0000 7fc6b72d6640 1 Processor -- start 2024-03-13T07:34:59.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.092+0000 7fc6b72d6640 1 -- start start 2024-03-13T07:34:59.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.092+0000 7fc6b72d6640 10 monclient: init 2024-03-13T07:34:59.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.092+0000 7fc6b72d6640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:59.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.092+0000 7fc6b72d6640 10 monclient: _add_conns ranks=[3,0,2,1,4] 2024-03-13T07:34:59.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.092+0000 7fc6b72d6640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc6b00b86c0 0x7fc6b013f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:59.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.092+0000 7fc6b72d6640 10 monclient(hunting): picked mon.noname-a con 0x7fc6b00b86c0 addr v2:172.21.15.132:3300/0 2024-03-13T07:34:59.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.092+0000 7fc6b72d6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc6b00b2390 addr v1:172.21.15.132:6789/0 2024-03-13T07:34:59.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.092+0000 7fc6b72d6640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b013fe90 0x7fc6b01462f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:59.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.092+0000 7fc6b72d6640 10 monclient(hunting): picked mon.noname-b con 0x7fc6b013fe90 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:59.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.092+0000 7fc6b72d6640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b00ab0a0 con 0x7fc6b00b2390 2024-03-13T07:34:59.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.092+0000 7fc6b72d6640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:59.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.092+0000 7fc6b72d6640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b00b8aa0 con 0x7fc6b00b86c0 2024-03-13T07:34:59.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.092+0000 7fc6b72d6640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:59.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.092+0000 7fc6b72d6640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b0140340 con 0x7fc6b013fe90 2024-03-13T07:34:59.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.092+0000 7fc6b72d6640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:59.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.092+0000 7fc6b72d6640 10 monclient(hunting): authenticate will time out at 1326.500861s 2024-03-13T07:34:59.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.093+0000 7fc6b5072640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc6b00b86c0 0x7fc6b013f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:59.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.093+0000 7fc6b5873640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b013fe90 0x7fc6b01462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:59.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.093+0000 7fc6b5072640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc6b00b86c0 0x7fc6b013f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34698/0 (socket says 172.21.15.132:34698) 2024-03-13T07:34:59.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.093+0000 7fc6b5072640 1 -- 172.21.15.132:0/1271977787 learned_addr learned my addr 172.21.15.132:0/1271977787 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:59.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.093+0000 7fc6b5072640 10 monclient(hunting): get_auth_request con 0x7fc6b00b86c0 auth_method 0 2024-03-13T07:34:59.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.093+0000 7fc6b5072640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:59.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.093+0000 7fc6b5072640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:59.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.093+0000 7fc6b5072640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:59.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.093+0000 7fc6b5873640 10 monclient(hunting): get_auth_request con 0x7fc6b013fe90 auth_method 0 2024-03-13T07:34:59.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.093+0000 7fc6b5873640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:59.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.093+0000 7fc6b5873640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:59.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.093+0000 7fc6b5873640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:59.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.094+0000 7fc6b5072640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:59.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.094+0000 7fc6b5072640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:59.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.094+0000 7fc6b5072640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:59.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.094+0000 7fc6b5873640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:59.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.094+0000 7fc6b5873640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:59.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.094+0000 7fc6b5873640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:59.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.096+0000 7fc6b5873640 10 monclient(hunting): handle_auth_done global_id 4343 payload 274 2024-03-13T07:34:59.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.096+0000 7fc6b5873640 1 -- 172.21.15.132:0/1271977787 >> v2:172.21.15.132:3300/0 conn(0x7fc6b00b86c0 msgr2=0x7fc6b013f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:59.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.096+0000 7fc6b5873640 1 --2- 172.21.15.132:0/1271977787 >> v2:172.21.15.132:3300/0 conn(0x7fc6b00b86c0 0x7fc6b013f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:59.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.096+0000 7fc6b5873640 1 -- 172.21.15.132:0/1271977787 >> v1:172.21.15.132:6789/0 conn(0x7fc6b00b2390 legacy=0x7fc6b0145be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:34:59.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.096+0000 7fc6b5873640 10 monclient: _finish_hunting 0 2024-03-13T07:34:59.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.096+0000 7fc6b5873640 1 monclient: found mon.noname-b 2024-03-13T07:34:59.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.096+0000 7fc6b5873640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:59.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.096+0000 7fc6b5873640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T07:34:59.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.096+0000 7fc6b5873640 1 -- 172.21.15.132:0/1271977787 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b00ab380 con 0x7fc6b013fe90 2024-03-13T07:34:59.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.098+0000 7fc6b5873640 1 --2- 172.21.15.132:0/1271977787 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b013fe90 0x7fc6b01462f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac0097a0 tx=0x7fc6ac02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8585b6ce056f4f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:59.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.101+0000 7fc6a7fff640 1 -- 172.21.15.132:0/1271977787 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0315a0 con 0x7fc6b013fe90 2024-03-13T07:34:59.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.101+0000 7fc6a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:59.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.101+0000 7fc6a7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:59.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.101+0000 7fc6a7fff640 10 monclient: dump: 2024-03-13T07:34:59.102 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:59.102 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:59.102 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:59.102 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:59.102 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:59.102 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:59.102 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:59.102 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:59.102 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:59.102 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:59.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.101+0000 7fc6a7fff640 10 monclient: _finish_auth 0 2024-03-13T07:34:59.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.101+0000 7fc6a7fff640 10 monclient: _check_auth_tickets 2024-03-13T07:34:59.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.101+0000 7fc6a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:59.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.101+0000 7fc6a7fff640 1 -- 172.21.15.132:0/1271977787 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac037070 con 0x7fc6b013fe90 2024-03-13T07:34:59.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.101+0000 7fc6a7fff640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:59.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.101+0000 7fc6a7fff640 1 -- 172.21.15.132:0/1271977787 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac031960 con 0x7fc6b013fe90 2024-03-13T07:34:59.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.101+0000 7fc6b72d6640 5 monclient: authenticate success, global_id 4343 2024-03-13T07:34:59.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.101+0000 7fc6b72d6640 10 monclient: get_monmap_and_config success 2024-03-13T07:34:59.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.101+0000 7fc6a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:59.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.101+0000 7fc6b72d6640 4 set_mon_vals no callback set 2024-03-13T07:34:59.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.101+0000 7fc6a7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:34:59.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.101+0000 7fc6a7fff640 10 monclient: dump: 2024-03-13T07:34:59.103 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:59.103 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:59.103 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:59.103 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:59.103 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:59.103 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:59.103 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:59.104 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:59.104 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:59.104 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:59.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.102+0000 7fc6b72d6640 10 monclient: shutdown 2024-03-13T07:34:59.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.102+0000 7fc6b72d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:59.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.102+0000 7fc6b72d6640 1 -- 172.21.15.132:0/1271977787 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b013fe90 msgr2=0x7fc6b01462f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:59.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.102+0000 7fc6b72d6640 1 --2- 172.21.15.132:0/1271977787 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b013fe90 0x7fc6b01462f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac0097a0 tx=0x7fc6ac02b840 comp rx=0 tx=0).stop 2024-03-13T07:34:59.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.102+0000 7fc6b72d6640 1 -- 172.21.15.132:0/1271977787 shutdown_connections 2024-03-13T07:34:59.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.102+0000 7fc6b72d6640 1 --2- 172.21.15.132:0/1271977787 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b013fe90 0x7fc6b01462f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:59.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.102+0000 7fc6b72d6640 1 --2- 172.21.15.132:0/1271977787 >> v2:172.21.15.132:3300/0 conn(0x7fc6b00b86c0 0x7fc6b013f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:59.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.102+0000 7fc6b72d6640 1 -- 172.21.15.132:0/1271977787 >> 172.21.15.132:0/1271977787 conn(0x7fc6b0066e20 msgr2=0x7fc6b0067230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:59.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.102+0000 7fc6b72d6640 1 -- 172.21.15.132:0/1271977787 shutdown_connections 2024-03-13T07:34:59.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.102+0000 7fc6b72d6640 1 -- 172.21.15.132:0/1271977787 wait complete. 2024-03-13T07:34:59.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.103+0000 7fc6b72d6640 10 monclient: build_initial_monmap 2024-03-13T07:34:59.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.103+0000 7fc6b72d6640 10 monclient: monmap: 2024-03-13T07:34:59.108 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T07:34:59.108 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:34:59.108 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T07:34:59.108 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T07:34:59.108 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T07:34:59.108 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:59.108 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:34:59.109 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:34:59.109 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:34:59.109 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:59.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.103+0000 7fc6b72d6640 1 librados: starting msgr at 2024-03-13T07:34:59.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.103+0000 7fc6b72d6640 1 librados: starting objecter 2024-03-13T07:34:59.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.103+0000 7fc6b72d6640 1 Processor -- start 2024-03-13T07:34:59.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 1 -- start start 2024-03-13T07:34:59.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 1 librados: setting wanted keys 2024-03-13T07:34:59.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 1 librados: calling monclient init 2024-03-13T07:34:59.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 10 monclient: init 2024-03-13T07:34:59.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 10 monclient: _reopen_session rank -1 2024-03-13T07:34:59.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 10 monclient: _add_conns ranks=[1,0,2] 2024-03-13T07:34:59.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6b00b2390 0x7fc6b0139380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:59.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 10 monclient(hunting): picked mon.noname-b con 0x7fc6b00b2390 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:34:59.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6b00b86c0 0x7fc6b012f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:59.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 10 monclient(hunting): picked mon.noname-a con 0x7fc6b00b86c0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:34:59.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b013fe90 0x7fc6b012f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:59.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 10 monclient(hunting): picked mon.noname-c con 0x7fc6b013fe90 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:34:59.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:59.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b0146830 con 0x7fc6b00b86c0 2024-03-13T07:34:59.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:59.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b00a9f90 con 0x7fc6b00b2390 2024-03-13T07:34:59.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 10 monclient(hunting): start opening mon connection 2024-03-13T07:34:59.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b014cf50 con 0x7fc6b013fe90 2024-03-13T07:34:59.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 10 monclient(hunting): _renew_subs 2024-03-13T07:34:59.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.104+0000 7fc6b72d6640 10 monclient(hunting): authenticate will time out at 1326.512663s 2024-03-13T07:34:59.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b4871640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6b00b86c0 0x7fc6b012f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:59.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b5873640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b013fe90 0x7fc6b012f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:59.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b5072640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6b00b2390 0x7fc6b0139380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:59.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b5873640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b013fe90 0x7fc6b012f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:57916/0 (socket says 172.21.15.132:57916) 2024-03-13T07:34:59.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b5873640 1 -- 172.21.15.132:0/3077411048 learned_addr learned my addr 172.21.15.132:0/3077411048 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:34:59.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b5873640 10 monclient(hunting): get_auth_request con 0x7fc6b013fe90 auth_method 0 2024-03-13T07:34:59.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b5873640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:59.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b5873640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:59.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b5873640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:59.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b4871640 10 monclient(hunting): get_auth_request con 0x7fc6b00b86c0 auth_method 0 2024-03-13T07:34:59.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b4871640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:59.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b4871640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:59.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b4871640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:59.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b5072640 10 monclient(hunting): get_auth_request con 0x7fc6b00b2390 auth_method 0 2024-03-13T07:34:59.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b5072640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:34:59.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b5072640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:34:59.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b5072640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:34:59.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b5873640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:59.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b5873640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:59.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.105+0000 7fc6b5873640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:59.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.106+0000 7fc6b5072640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:59.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.106+0000 7fc6b5072640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:59.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.106+0000 7fc6b5072640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:59.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.106+0000 7fc6b4871640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:34:59.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.106+0000 7fc6b4871640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:34:59.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.106+0000 7fc6b4871640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:34:59.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.108+0000 7fc6b5072640 10 monclient(hunting): handle_auth_done global_id 4366 payload 824 2024-03-13T07:34:59.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.108+0000 7fc6b5072640 1 -- 172.21.15.132:0/3077411048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b013fe90 msgr2=0x7fc6b012f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:59.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.108+0000 7fc6b5072640 1 --2- 172.21.15.132:0/3077411048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b013fe90 0x7fc6b012f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:59.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.108+0000 7fc6b5072640 1 -- 172.21.15.132:0/3077411048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6b00b86c0 msgr2=0x7fc6b012f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:59.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.108+0000 7fc6b5072640 1 --2- 172.21.15.132:0/3077411048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6b00b86c0 0x7fc6b012f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:59.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.108+0000 7fc6b5072640 10 monclient: _finish_hunting 0 2024-03-13T07:34:59.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.108+0000 7fc6b5072640 1 monclient: found mon.noname-b 2024-03-13T07:34:59.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.108+0000 7fc6b5072640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:34:59.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.108+0000 7fc6b5072640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T07:34:59.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.108+0000 7fc6b5072640 1 -- 172.21.15.132:0/3077411048 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b0138bf0 con 0x7fc6b00b2390 2024-03-13T07:34:59.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.111+0000 7fc6b5072640 1 --2- 172.21.15.132:0/3077411048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6b00b2390 0x7fc6b0139380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800efc0 tx=0x7fc6a800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:59.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.111+0000 7fc6a5ffb640 10 client.?.objecter ms_handle_connect 0x7fc6b00b2390 2024-03-13T07:34:59.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.111+0000 7fc6a5ffb640 10 client.?.objecter resend_mon_ops 2024-03-13T07:34:59.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6a5ffb640 1 -- 172.21.15.132:0/3077411048 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a800cda0 con 0x7fc6b00b2390 2024-03-13T07:34:59.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6a5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:59.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6a5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T07:34:59.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6a5ffb640 10 monclient: dump: 2024-03-13T07:34:59.115 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:59.115 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:59.115 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:59.115 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:59.115 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:59.116 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:59.116 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:59.116 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:59.116 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:59.116 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:59.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6a5ffb640 10 monclient: _finish_auth 0 2024-03-13T07:34:59.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6a5ffb640 10 monclient: _check_auth_tickets 2024-03-13T07:34:59.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6a5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:34:59.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6a5ffb640 1 -- 172.21.15.132:0/3077411048 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a8019070 con 0x7fc6b00b2390 2024-03-13T07:34:59.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6b72d6640 5 monclient: authenticate success, global_id 4366 2024-03-13T07:34:59.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6b72d6640 10 monclient: _renew_subs 2024-03-13T07:34:59.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6b72d6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:59.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6b72d6640 1 -- 172.21.15.132:0/3077411048 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b01398c0 con 0x7fc6b00b2390 2024-03-13T07:34:59.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6a5ffb640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:34:59.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6a5ffb640 1 -- 172.21.15.132:0/3077411048 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8014680 con 0x7fc6b00b2390 2024-03-13T07:34:59.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6a5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:34:59.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6a5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T07:34:59.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6a5ffb640 10 monclient: dump: 2024-03-13T07:34:59.117 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T07:34:59.117 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:34:59.117 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:59.117 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:34:59.117 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T07:34:59.117 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T07:34:59.117 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:34:59.117 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:34:59.117 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:34:59.117 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T07:34:59.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6a6ffd640 4 set_mon_vals no callback set 2024-03-13T07:34:59.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6b72d6640 10 client.4366.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:34:59.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6b72d6640 10 monclient: _renew_subs 2024-03-13T07:34:59.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6b72d6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:59.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6b72d6640 1 -- 172.21.15.132:0/3077411048 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b0137990 con 0x7fc6b00b2390 2024-03-13T07:34:59.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.114+0000 7fc6b72d6640 1 librados: init done 2024-03-13T07:34:59.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.115+0000 7fc6b72d6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:59.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.115+0000 7fc6b72d6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:59.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.115+0000 7fc6b72d6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:59.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.115+0000 7fc6b72d6640 1 -- 172.21.15.132:0/3077411048 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b0139c30 con 0x7fc6b00b2390 2024-03-13T07:34:59.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.132+0000 7fc6a5ffb640 1 -- 172.21.15.132:0/3077411048 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fc6a8005d10 con 0x7fc6b00b2390 2024-03-13T07:34:59.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.133+0000 7fc6a5ffb640 1 --2- 172.21.15.132:0/3077411048 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fc69406d5b0 0x7fc69406fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:34:59.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.133+0000 7fc6b4871640 1 --2- 172.21.15.132:0/3077411048 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fc69406d5b0 0x7fc69406fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:34:59.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.133+0000 7fc6b4871640 10 monclient: get_auth_request con 0x7fc69406d5b0 auth_method 0 2024-03-13T07:34:59.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.133+0000 7fc6b4871640 1 --2- 172.21.15.132:0/3077411048 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fc69406d5b0 0x7fc69406fa70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0146570 tx=0x7fc6a0007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:34:59.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.133+0000 7fc6a5ffb640 10 client.4366.objecter ms_handle_connect 0x7fc69406d5b0 2024-03-13T07:34:59.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.135+0000 7fc6a5ffb640 1 -- 172.21.15.132:0/3077411048 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc6a8057f00 con 0x7fc6b00b2390 2024-03-13T07:34:59.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.135+0000 7fc6a5ffb640 10 client.4366.objecter ms_dispatch 0x7fc6b0000b90 osd_map(26..26 src has 1..26) v4 2024-03-13T07:34:59.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.135+0000 7fc6a5ffb640 3 client.4366.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:34:59.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.135+0000 7fc6a5ffb640 3 client.4366.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:34:59.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.135+0000 7fc6a5ffb640 20 client.4366.objecter dump_active .. 0 homeless 2024-03-13T07:34:59.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.409+0000 7fc6a5ffb640 1 -- 172.21.15.132:0/3077411048 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fc6a8010040 con 0x7fc6b00b2390 2024-03-13T07:34:59.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.409+0000 7fc6a5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T07:34:59.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.409+0000 7fc6a5ffb640 10 monclient: _finish_command 1 = system:0 2024-03-13T07:34:59.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.635+0000 7fc6b72d6640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2024-03-13T07:34:59.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.635+0000 7fc6b72d6640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2024-03-13T07:34:59.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.635+0000 7fc6b72d6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T07:34:59.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.635+0000 7fc6b72d6640 1 -- 172.21.15.132:0/3077411048 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc6b004f000 con 0x7fc6b00b2390 2024-03-13T07:34:59.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.640+0000 7fc6a5ffb640 1 -- 172.21.15.132:0/3077411048 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc6a805ee20 con 0x7fc6b00b2390 2024-03-13T07:34:59.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.640+0000 7fc6a5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2024-03-13T07:34:59.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.640+0000 7fc6a5ffb640 10 monclient: _finish_command 2 = system:0 2024-03-13T07:34:59.640 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T07:34:59.640 INFO:teuthology.orchestra.run.smithi132.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-13T07:34:59.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 10 librados: watch_flush enter 2024-03-13T07:34:59.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 10 librados: watch_flush exit 2024-03-13T07:34:59.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 20 client.4366.objecter shutdown clearing up homeless session... 2024-03-13T07:34:59.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 10 client.4366.objecter successfully canceled tick 2024-03-13T07:34:59.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 1 -- 172.21.15.132:0/3077411048 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fc69406d5b0 msgr2=0x7fc69406fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:59.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 1 --2- 172.21.15.132:0/3077411048 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fc69406d5b0 0x7fc69406fa70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0146570 tx=0x7fc6a0007e30 comp rx=0 tx=0).stop 2024-03-13T07:34:59.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 10 monclient: shutdown 2024-03-13T07:34:59.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:34:59.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 1 -- 172.21.15.132:0/3077411048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6b00b2390 msgr2=0x7fc6b0139380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:34:59.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 1 --2- 172.21.15.132:0/3077411048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6b00b2390 0x7fc6b0139380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800efc0 tx=0x7fc6a800c490 comp rx=0 tx=0).stop 2024-03-13T07:34:59.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 1 -- 172.21.15.132:0/3077411048 shutdown_connections 2024-03-13T07:34:59.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 1 --2- 172.21.15.132:0/3077411048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6b013fe90 0x7fc6b012f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:59.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 1 --2- 172.21.15.132:0/3077411048 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fc69406d5b0 0x7fc69406fa70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:59.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 1 --2- 172.21.15.132:0/3077411048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6b00b86c0 0x7fc6b012f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:59.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 1 --2- 172.21.15.132:0/3077411048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6b00b2390 0x7fc6b0139380 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:34:59.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 1 -- 172.21.15.132:0/3077411048 >> 172.21.15.132:0/3077411048 conn(0x7fc6b0066e20 msgr2=0x7fc6b00b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:34:59.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 1 -- 172.21.15.132:0/3077411048 shutdown_connections 2024-03-13T07:34:59.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 1 -- 172.21.15.132:0/3077411048 wait complete. 2024-03-13T07:34:59.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T07:34:59.642+0000 7fc6b72d6640 1 librados: shutdown 2024-03-13T07:34:59.653 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-13T07:34:59.653 INFO:teuthology.run_tasks:Running task workunit... 2024-03-13T07:34:59.667 INFO:tasks.workunit:Pulling workunits from ref b7c579c9c3ed531070a6aa96f74f80b53f80de32 2024-03-13T07:34:59.667 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-13T07:34:59.667 DEBUG:teuthology.orchestra.run.smithi132:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-13T07:34:59.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T07:34:59.682 INFO:teuthology.orchestra.run.smithi132.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-13T07:34:59.682 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-13T07:34:59.737 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-13T07:34:59.738 DEBUG:teuthology.orchestra.run.smithi132:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-13T07:34:59.792 INFO:tasks.workunit:timeout=6h 2024-03-13T07:34:59.792 INFO:tasks.workunit:cleanup=True 2024-03-13T07:34:59.792 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout b7c579c9c3ed531070a6aa96f74f80b53f80de32 2024-03-13T07:34:59.848 INFO:tasks.workunit.client.0.smithi132.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-13T07:35:05.409 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:01:29.299 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:05.418 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:01:29.308 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:05.460 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:01:29.350 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:05.463 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:01:29.334 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:05.467 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:01:29.338 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:05.476 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:01:29.367 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:05.478 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:01:29.368 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:05.505 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:01:29.375 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:05.515 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:01:29.405 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:25.437 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:01:49.327 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:25.499 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:01:49.369 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:25.501 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:01:49.371 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:25.518 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:01:49.408 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:25.523 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:01:49.393 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:25.575 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:01:49.465 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:25.598 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:01:49.489 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:25.607 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:01:49.497 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:25.631 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:01:49.521 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:45.536 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:02:09.407 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:45.551 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:02:09.441 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:45.555 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:02:09.446 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:45.575 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:02:09.446 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:45.593 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:02:09.463 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:45.632 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:02:09.523 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:45.634 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:02:09.524 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:45.644 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:02:09.535 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:45.668 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:02:09.559 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:35:50.659 INFO:tasks.workunit.client.0.smithi132.stderr:Note: switching to 'b7c579c9c3ed531070a6aa96f74f80b53f80de32'. 2024-03-13T07:35:50.659 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-03-13T07:35:50.659 INFO:tasks.workunit.client.0.smithi132.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-13T07:35:50.659 INFO:tasks.workunit.client.0.smithi132.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-13T07:35:50.659 INFO:tasks.workunit.client.0.smithi132.stderr:state without impacting any branches by switching back to a branch. 2024-03-13T07:35:50.659 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-03-13T07:35:50.659 INFO:tasks.workunit.client.0.smithi132.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-13T07:35:50.659 INFO:tasks.workunit.client.0.smithi132.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-13T07:35:50.659 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-03-13T07:35:50.660 INFO:tasks.workunit.client.0.smithi132.stderr: git switch -c 2024-03-13T07:35:50.660 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-03-13T07:35:50.660 INFO:tasks.workunit.client.0.smithi132.stderr:Or undo this operation with: 2024-03-13T07:35:50.660 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-03-13T07:35:50.660 INFO:tasks.workunit.client.0.smithi132.stderr: git switch - 2024-03-13T07:35:50.660 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-03-13T07:35:50.660 INFO:tasks.workunit.client.0.smithi132.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-13T07:35:50.660 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-03-13T07:35:50.660 INFO:tasks.workunit.client.0.smithi132.stderr:HEAD is now at b7c579c9c3e Merge branch 'wip-refactor-ec-get_chunk_size' of https://github.com/rzarzynski/ceph into wip-yuri8-testing-2024-03-11-1138 2024-03-13T07:35:50.663 DEBUG:teuthology.orchestra.run.smithi132:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-13T07:35:50.726 INFO:tasks.workunit.client.0.smithi132.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-13T07:35:50.728 INFO:tasks.workunit.client.0.smithi132.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-13T07:35:50.728 INFO:tasks.workunit.client.0.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-13T07:35:51.365 INFO:tasks.workunit.client.0.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-13T07:35:51.431 INFO:tasks.workunit.client.0.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-13T07:35:51.470 INFO:tasks.workunit.client.0.smithi132.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-13T07:35:51.472 INFO:tasks.workunit.client.0.smithi132.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-13T07:35:51.472 INFO:tasks.workunit.client.0.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-13T07:35:51.512 INFO:tasks.workunit.client.0.smithi132.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-13T07:35:51.515 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-13T07:35:51.515 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-13T07:35:51.571 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-03-13T07:35:51.572 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-03-13T07:35:51.572 DEBUG:teuthology.orchestra.run.smithi132:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b7c579c9c3ed531070a6aa96f74f80b53f80de32 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-03-13T07:35:51.629 INFO:tasks.workunit.client.0.smithi132.stderr:+ parallel=1 2024-03-13T07:35:51.630 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' '' = --serial ']' 2024-03-13T07:35:51.630 INFO:tasks.workunit.client.0.smithi132.stderr:+ crimson=0 2024-03-13T07:35:51.630 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' '' = --crimson ']' 2024-03-13T07:35:51.630 INFO:tasks.workunit.client.0.smithi132.stderr:+ color= 2024-03-13T07:35:51.630 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -t 1 ']' 2024-03-13T07:35:51.630 INFO:tasks.workunit.client.0.smithi132.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-03-13T07:35:51.630 INFO:tasks.workunit.client.0.smithi132.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-13T07:35:51.630 INFO:tasks.workunit.client.0.smithi132.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-13T07:35:51.631 INFO:tasks.workunit.client.0.smithi132.stderr:+ declare -A pids 2024-03-13T07:35:51.631 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.632 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.632 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_aio 2024-03-13T07:35:51.632 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_aio' 2024-03-13T07:35:51.632 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_aio 2024-03-13T07:35:51.632 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.632 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_aio 2024-03-13T07:35:51.633 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40760 2024-03-13T07:35:51.633 INFO:tasks.workunit.client.0.smithi132.stdout:test api_aio on pid 40760 2024-03-13T07:35:51.636 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_aio on pid 40760' 2024-03-13T07:35:51.636 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=40760 2024-03-13T07:35:51.636 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.636 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.636 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-03-13T07:35:51.636 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_aio_pp 2024-03-13T07:35:51.636 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_aio_pp' 2024-03-13T07:35:51.637 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_aio_pp 2024-03-13T07:35:51.637 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.637 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.637 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.637 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.637 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.637 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.637 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.637 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.637 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.637 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.637 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.637 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.638 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.638 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.638 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.638 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.638 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.638 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.638 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.638 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.638 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.638 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.638 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.638 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.639 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.639 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.639 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_aio_pp 2024-03-13T07:35:51.639 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40767 2024-03-13T07:35:51.639 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_aio_pp on pid 40767' 2024-03-13T07:35:51.639 INFO:tasks.workunit.client.0.smithi132.stdout:test api_aio_pp on pid 40767 2024-03-13T07:35:51.643 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=40767 2024-03-13T07:35:51.643 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.643 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-03-13T07:35:51.643 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.643 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_io 2024-03-13T07:35:51.643 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_io' 2024-03-13T07:35:51.643 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_io 2024-03-13T07:35:51.643 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.644 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.644 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.644 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.644 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.644 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.644 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.644 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.644 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.644 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.644 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.644 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.644 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.644 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.645 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.645 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.645 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.645 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.645 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.645 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.645 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.645 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.645 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.645 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.645 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.645 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.646 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.646 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.646 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.646 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.646 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.646 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.646 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.646 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.646 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.646 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.646 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.646 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.647 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.647 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.647 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.647 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.647 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.647 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.647 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.647 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.647 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_io 2024-03-13T07:35:51.647 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40776 2024-03-13T07:35:51.647 INFO:tasks.workunit.client.0.smithi132.stdout:test api_io on pid 40776 2024-03-13T07:35:51.653 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_io on pid 40776' 2024-03-13T07:35:51.653 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=40776 2024-03-13T07:35:51.653 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.653 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-03-13T07:35:51.653 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.653 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_io_pp 2024-03-13T07:35:51.653 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_io_pp' 2024-03-13T07:35:51.654 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_io_pp 2024-03-13T07:35:51.654 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.654 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.654 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.654 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.654 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.654 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.654 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.654 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.654 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.654 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.654 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.654 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.654 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.655 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.655 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.655 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.655 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.655 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.655 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.655 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.655 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.655 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.655 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.655 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.655 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.655 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.655 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.656 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.656 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.656 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.656 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.656 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.656 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.656 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.656 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.656 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.656 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.656 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.656 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.656 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.656 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.656 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.657 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.657 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.657 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.657 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.657 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.657 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.657 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.657 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.657 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.657 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.657 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.657 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.657 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.657 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_io_pp 2024-03-13T07:35:51.657 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40788 2024-03-13T07:35:51.658 INFO:tasks.workunit.client.0.smithi132.stdout:test api_io_pp on pid 40788 2024-03-13T07:35:51.664 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_io_pp on pid 40788' 2024-03-13T07:35:51.664 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=40788 2024-03-13T07:35:51.664 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.665 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.665 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-03-13T07:35:51.665 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_asio 2024-03-13T07:35:51.665 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_asio' 2024-03-13T07:35:51.665 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_asio 2024-03-13T07:35:51.665 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.665 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.665 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.665 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.665 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.665 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.665 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.665 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.666 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.666 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.666 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.666 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.666 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.666 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.666 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.666 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.666 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.666 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.666 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.666 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.666 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.666 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.667 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.667 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.667 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.667 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.667 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.667 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.667 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.667 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.667 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.667 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.667 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.667 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.667 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.668 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.668 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.668 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.668 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.668 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.668 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.668 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.668 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.668 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.669 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.669 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.669 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.669 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.669 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.669 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.669 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.669 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.669 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.669 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.669 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.669 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.669 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.669 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.670 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.670 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.670 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.670 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.670 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.670 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.670 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.670 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.670 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.670 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.670 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.671 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.671 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.671 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.671 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.671 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.671 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.671 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.671 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.671 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.671 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.671 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_asio 2024-03-13T07:35:51.671 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40804 2024-03-13T07:35:51.671 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_asio on pid 40804' 2024-03-13T07:35:51.671 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-03-13T07:35:51.672 INFO:tasks.workunit.client.0.smithi132.stdout:test api_asio on pid 40804 2024-03-13T07:35:51.683 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=40804 2024-03-13T07:35:51.683 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.683 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.683 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_list 2024-03-13T07:35:51.683 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.683 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.683 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.684 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.684 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.684 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.684 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.684 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_list' 2024-03-13T07:35:51.684 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.684 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.684 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.684 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.684 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.684 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.684 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.684 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.685 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.685 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.685 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.685 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.685 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.685 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.685 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.685 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.685 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.685 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.685 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.686 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.686 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.686 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.686 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.686 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.686 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.686 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.686 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.686 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.686 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.686 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.686 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_list 2024-03-13T07:35:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.688 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.688 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.688 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.688 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.688 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.688 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.688 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.688 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.688 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.688 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.688 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.688 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.688 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.688 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.689 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.689 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.689 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.689 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.689 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.689 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.689 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.689 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.689 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.689 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.690 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.690 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.690 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.690 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.690 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.690 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.690 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.690 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.690 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.690 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.690 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.690 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.690 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.691 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.691 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.691 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.691 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.691 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.691 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/40760/exe 2024-03-13T07:35:51.691 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.691 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.691 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.691 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.691 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.691 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.691 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.691 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.692 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.692 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.692 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.692 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.692 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.692 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.692 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.692 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.692 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.692 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.692 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.692 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.692 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.692 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.693 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.693 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.693 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.693 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.693 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.693 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.693 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.693 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.693 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.693 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.693 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.693 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.693 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.693 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_aio.log 2024-03-13T07:35:51.693 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.694 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.694 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2024-03-13T07:35:51.694 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_aio: /' 2024-03-13T07:35:51.694 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_list 2024-03-13T07:35:51.694 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.694 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.694 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.694 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.694 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.694 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40826 2024-03-13T07:35:51.694 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.694 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.694 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.695 INFO:tasks.workunit.client.0.smithi132.stdout:test api_list on pid 40826 2024-03-13T07:35:51.709 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_list on pid 40826' 2024-03-13T07:35:51.709 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.709 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=40826 2024-03-13T07:35:51.709 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.710 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.710 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.710 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.710 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.710 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.710 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.710 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.710 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.710 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.710 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.710 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.710 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.710 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.710 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.711 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.711 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.711 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.711 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.711 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-03-13T07:35:51.711 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.711 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.711 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.711 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.711 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.711 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.711 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.711 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.711 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.711 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.712 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.712 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.712 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.712 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.712 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.712 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.712 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.712 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_lock 2024-03-13T07:35:51.712 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.712 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.712 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.712 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.712 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.712 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.712 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.712 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.713 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.713 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_lock' 2024-03-13T07:35:51.713 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.713 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.713 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.713 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.713 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.713 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.713 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.713 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.713 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.713 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.713 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.713 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.713 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.714 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.714 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.714 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.714 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.714 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.714 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.714 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.714 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.714 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.714 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.714 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.714 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.714 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.714 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.715 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.715 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.715 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.715 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.715 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.715 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.715 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.715 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.715 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.715 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.715 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_lock 2024-03-13T07:35:51.715 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.715 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.715 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.715 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.715 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.716 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.716 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.716 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.716 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.716 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.716 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.716 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.716 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.716 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.716 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.716 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.716 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.716 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.716 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.717 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.717 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/40767/exe 2024-03-13T07:35:51.717 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.717 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.717 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.717 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.717 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.717 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.717 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.717 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.717 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.717 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.717 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.717 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.717 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.717 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.718 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.718 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.718 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.718 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.718 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.718 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.718 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.718 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.718 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.718 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.718 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.718 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.718 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.718 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.719 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.719 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.719 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.719 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.719 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.719 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.719 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.719 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.719 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.719 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.719 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.719 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.719 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.719 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.719 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.720 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.720 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.720 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.720 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.720 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.720 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.720 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.720 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.720 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.720 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.720 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.720 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.720 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.720 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.720 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.721 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.721 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.721 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.721 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.721 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2024-03-13T07:35:51.721 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.721 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.721 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/40776/exe 2024-03-13T07:35:51.721 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.721 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.721 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.721 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_aio_pp: /' 2024-03-13T07:35:51.721 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.721 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.722 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.722 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.722 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.722 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.722 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.722 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.722 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.722 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.722 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-03-13T07:35:51.722 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.722 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_lock 2024-03-13T07:35:51.722 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.722 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40854 2024-03-13T07:35:51.723 INFO:tasks.workunit.client.0.smithi132.stdout:test api_lock on pid 40854 2024-03-13T07:35:51.732 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_lock on pid 40854' 2024-03-13T07:35:51.732 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=40854 2024-03-13T07:35:51.732 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.732 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.732 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.732 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.732 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.732 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/40788/exe 2024-03-13T07:35:51.732 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.732 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.732 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.732 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_lock_pp 2024-03-13T07:35:51.733 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.733 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.733 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.733 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.733 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.733 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_lock_pp' 2024-03-13T07:35:51.733 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.733 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.733 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.733 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.733 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.733 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.733 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.733 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.734 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.734 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.734 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.734 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.734 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.734 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.734 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.734 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.734 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.734 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_lock_pp 2024-03-13T07:35:51.734 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-03-13T07:35:51.734 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_io: /' 2024-03-13T07:35:51.734 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2024-03-13T07:35:51.734 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_io.log 2024-03-13T07:35:51.734 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.735 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.735 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.735 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.735 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.735 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.735 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.735 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.735 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.735 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.735 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.735 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2024-03-13T07:35:51.735 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.735 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.735 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_io_pp: /' 2024-03-13T07:35:51.736 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.736 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.736 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.736 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.736 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.736 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.736 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.736 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.736 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.736 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.736 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.736 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.737 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.737 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.737 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.737 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.737 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.737 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.737 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_lock_pp 2024-03-13T07:35:51.737 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.737 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.737 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.737 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40870 2024-03-13T07:35:51.737 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_lock_pp on pid 40870' 2024-03-13T07:35:51.738 INFO:tasks.workunit.client.0.smithi132.stdout:test api_lock_pp on pid 40870 2024-03-13T07:35:51.745 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=40870 2024-03-13T07:35:51.745 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.745 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.745 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.745 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.745 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.745 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.745 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.745 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.745 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.745 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.746 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.746 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.746 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.746 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.746 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-03-13T07:35:51.746 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.746 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.746 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.746 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.746 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.746 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_misc 2024-03-13T07:35:51.746 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_misc' 2024-03-13T07:35:51.746 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.746 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.747 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.747 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.747 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.747 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.747 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_misc 2024-03-13T07:35:51.747 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.747 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.747 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.747 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.747 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.747 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.747 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.747 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.747 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.748 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.748 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.748 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.748 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.748 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.748 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.748 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.748 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.748 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.748 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.748 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.748 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.748 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.748 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.748 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.749 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.749 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.749 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.749 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.749 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.749 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.749 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.749 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.749 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.749 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.749 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.749 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.749 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.749 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.749 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.749 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.750 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.750 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.750 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.750 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.750 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.750 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.750 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.750 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.750 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.750 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.750 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.750 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.750 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.750 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.751 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.751 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.751 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.751 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.751 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.751 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.751 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.751 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_misc 2024-03-13T07:35:51.751 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.751 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.751 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40886 2024-03-13T07:35:51.751 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_misc on pid 40886' 2024-03-13T07:35:51.752 INFO:tasks.workunit.client.0.smithi132.stdout:test api_misc on pid 40886 2024-03-13T07:35:51.759 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=40886 2024-03-13T07:35:51.759 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.759 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.759 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-03-13T07:35:51.759 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-03-13T07:35:51.759 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.759 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.759 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.759 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.760 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.760 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.760 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.760 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.760 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.760 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.760 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.760 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.760 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.760 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.760 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.760 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.760 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.761 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.761 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.761 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.761 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.761 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.761 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.761 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.761 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.761 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.761 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.761 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.761 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_misc_pp 2024-03-13T07:35:51.761 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.761 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.761 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_misc_pp' 2024-03-13T07:35:51.762 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.762 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.762 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.762 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.762 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.762 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.762 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.762 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.762 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.762 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.762 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.762 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.762 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.762 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.763 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.763 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.763 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.763 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.763 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.763 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.763 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.763 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.763 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.763 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.763 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.763 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.763 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.763 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.763 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_misc_pp 2024-03-13T07:35:51.763 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.764 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.764 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.764 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.764 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.764 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.764 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.764 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.764 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.764 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.764 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.764 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.764 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.764 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.764 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.765 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.765 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.765 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.765 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.765 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.765 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.765 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.765 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.765 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.765 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.765 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.765 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.765 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.765 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.765 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.765 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.766 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.766 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.766 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.766 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.766 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.766 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.766 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.766 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.766 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.766 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.766 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/40826/exe 2024-03-13T07:35:51.766 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/40804/exe 2024-03-13T07:35:51.766 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.766 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.767 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.767 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.767 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.767 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.767 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.767 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.767 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.767 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.767 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.767 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.767 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.767 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_misc_pp 2024-03-13T07:35:51.767 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.767 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.767 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40905 2024-03-13T07:35:51.768 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_misc_pp on pid 40905' 2024-03-13T07:35:51.768 INFO:tasks.workunit.client.0.smithi132.stdout:test api_misc_pp on pid 40905 2024-03-13T07:35:51.775 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=40905 2024-03-13T07:35:51.776 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.776 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.776 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.776 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.776 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.776 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.776 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.776 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.776 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.776 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.776 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.776 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.776 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.776 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.777 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.777 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.777 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.777 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_list: /' 2024-03-13T07:35:51.777 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.777 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_tier_pp 2024-03-13T07:35:51.777 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-03-13T07:35:51.777 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_tier_pp' 2024-03-13T07:35:51.777 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.777 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.777 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.777 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.777 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.777 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.777 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.778 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.778 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.778 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.778 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.778 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.778 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2024-03-13T07:35:51.778 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.778 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.778 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.778 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.778 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.778 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.778 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.778 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.778 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.779 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.779 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.779 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.779 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.779 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.779 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.779 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.779 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.779 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.779 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.779 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2024-03-13T07:35:51.779 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.779 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.779 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.780 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.780 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.780 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.780 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_tier_pp 2024-03-13T07:35:51.780 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.780 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_list.log 2024-03-13T07:35:51.780 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.780 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.780 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.780 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.780 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.780 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.780 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.780 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.780 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.780 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.781 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.781 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.781 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.781 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.781 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.781 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.781 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.781 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.781 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.781 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.781 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.781 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.781 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.781 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.781 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.781 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_asio.log 2024-03-13T07:35:51.782 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_tier_pp 2024-03-13T07:35:51.782 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40922 2024-03-13T07:35:51.782 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_tier_pp on pid 40922' 2024-03-13T07:35:51.782 INFO:tasks.workunit.client.0.smithi132.stdout:test api_tier_pp on pid 40922 2024-03-13T07:35:51.788 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=40922 2024-03-13T07:35:51.788 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.788 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.788 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.788 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.788 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.788 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.788 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.788 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.788 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.788 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.788 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.789 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.789 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.789 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.789 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.789 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.789 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.789 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.789 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.789 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.789 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.789 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.789 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.789 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.789 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.789 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.790 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.790 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.790 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.790 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.790 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.790 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.790 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.790 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.790 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.790 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.790 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.790 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.790 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.790 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.791 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.791 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.791 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.791 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.791 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.791 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_pool 2024-03-13T07:35:51.791 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_pool' 2024-03-13T07:35:51.791 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.791 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_asio: /' 2024-03-13T07:35:51.791 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-03-13T07:35:51.791 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.791 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.791 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_pool 2024-03-13T07:35:51.791 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.791 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.791 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.792 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.792 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.792 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.792 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_pool 2024-03-13T07:35:51.792 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40938 2024-03-13T07:35:51.792 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_pool on pid 40938' 2024-03-13T07:35:51.792 INFO:tasks.workunit.client.0.smithi132.stdout:test api_pool on pid 40938 2024-03-13T07:35:51.813 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=40938 2024-03-13T07:35:51.813 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.814 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.814 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.814 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.814 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.814 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.814 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.814 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.814 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.814 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.814 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.814 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.814 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.814 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.814 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.815 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.815 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.815 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.815 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.815 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.815 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.815 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.815 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.815 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.815 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.815 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.815 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.815 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.815 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_snapshots 2024-03-13T07:35:51.815 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.816 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.816 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_snapshots' 2024-03-13T07:35:51.816 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.816 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.816 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.816 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.816 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.816 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.816 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.816 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.816 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.816 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.816 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.816 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-03-13T07:35:51.816 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/40870/exe 2024-03-13T07:35:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2024-03-13T07:35:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_snapshots 2024-03-13T07:35:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-03-13T07:35:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_lock_pp: /' 2024-03-13T07:35:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/40886/exe 2024-03-13T07:35:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2024-03-13T07:35:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/40854/exe 2024-03-13T07:35:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_misc.log 2024-03-13T07:35:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_snapshots 2024-03-13T07:35:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41012 2024-03-13T07:35:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_snapshots on pid 41012' 2024-03-13T07:35:51.832 INFO:tasks.workunit.client.0.smithi132.stdout:test api_snapshots on pid 41012 2024-03-13T07:35:51.841 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41012 2024-03-13T07:35:51.841 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.841 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.841 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-03-13T07:35:51.841 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.841 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.841 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.841 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.841 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.842 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.842 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.842 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.842 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.842 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.842 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.842 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.842 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.842 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.842 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.842 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.842 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.842 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.842 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.843 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.843 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.843 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.843 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.843 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.843 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.843 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.843 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.843 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.843 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.843 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.843 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.843 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.843 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_misc: /' 2024-03-13T07:35:51.843 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/40905/exe 2024-03-13T07:35:51.844 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.844 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.844 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.844 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.844 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.844 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.844 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.844 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.844 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.844 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.844 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.844 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.844 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.844 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.845 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.845 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.845 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.845 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.845 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.845 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.845 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.845 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.845 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.845 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.845 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.845 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.845 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.845 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.845 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.845 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.846 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.846 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.846 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.846 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.846 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.846 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.846 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.846 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.846 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.846 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.846 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.846 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.846 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.846 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.847 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.847 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.847 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.847 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.847 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.847 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.847 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.847 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.847 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_misc_pp: /' 2024-03-13T07:35:51.847 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2024-03-13T07:35:51.847 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.847 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.847 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.847 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.847 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.848 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-03-13T07:35:51.848 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_lock.log 2024-03-13T07:35:51.848 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_lock: /' 2024-03-13T07:35:51.848 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_snapshots_pp 2024-03-13T07:35:51.848 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_snapshots_pp' 2024-03-13T07:35:51.848 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.848 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_snapshots_pp 2024-03-13T07:35:51.848 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_snapshots_pp 2024-03-13T07:35:51.848 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41034 2024-03-13T07:35:51.848 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_snapshots_pp on pid 41034' 2024-03-13T07:35:51.848 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-03-13T07:35:51.849 INFO:tasks.workunit.client.0.smithi132.stdout:test api_snapshots_pp on pid 41034 2024-03-13T07:35:51.852 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41034 2024-03-13T07:35:51.852 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.852 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.852 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_stat 2024-03-13T07:35:51.852 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2024-03-13T07:35:51.852 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_stat' 2024-03-13T07:35:51.852 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.853 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.853 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.853 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.853 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.853 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.853 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.853 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.853 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.853 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.853 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.853 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.853 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_stat 2024-03-13T07:35:51.853 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.853 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.853 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.854 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.854 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.854 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.854 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.854 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.854 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.854 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.854 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.854 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.854 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.854 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.854 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.854 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.854 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.854 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.855 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.855 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.855 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.855 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.855 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.855 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.855 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.855 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.855 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.855 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.855 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.855 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.855 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.855 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.855 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.856 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.856 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.856 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_stat 2024-03-13T07:35:51.856 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41054 2024-03-13T07:35:51.856 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_stat on pid 41054' 2024-03-13T07:35:51.856 INFO:tasks.workunit.client.0.smithi132.stdout:test api_stat on pid 41054 2024-03-13T07:35:51.862 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41054 2024-03-13T07:35:51.862 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.862 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.863 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.863 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.863 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.863 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.863 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.863 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.863 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.863 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.863 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.863 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.863 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.863 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.863 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.863 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.863 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.864 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.864 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.864 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.864 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.864 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.864 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.864 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.864 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.864 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.864 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.864 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.864 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.864 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.864 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.864 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.865 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.865 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.865 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.865 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.865 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.865 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.865 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.865 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.865 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.865 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.865 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.865 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.865 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.865 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.865 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.865 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.866 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.866 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.866 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.866 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.866 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.866 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.866 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.866 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.866 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_stat_pp 2024-03-13T07:35:51.866 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_stat_pp' 2024-03-13T07:35:51.866 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_stat_pp 2024-03-13T07:35:51.866 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-03-13T07:35:51.866 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.866 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.866 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.867 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.867 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.867 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.867 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.867 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.867 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.867 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.867 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.867 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.867 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.867 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.867 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.867 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-03-13T07:35:51.868 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [----------] Global test environment set-up. 2024-03-13T07:35:51.868 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [----------] 12 tests from AsioRados 2024-03-13T07:35:51.868 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.746+0000 7f71e0bf5780 10 monclient: get_monmap_and_config 2024-03-13T07:35:51.868 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.746+0000 7f71e0bf5780 10 monclient: build_initial_monmap 2024-03-13T07:35:51.868 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.746+0000 7f71e0bf5780 10 monclient: monmap: 2024-03-13T07:35:51.868 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: epoch 0 2024-03-13T07:35:51.868 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:35:51.868 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: last_changed 0.000000 2024-03-13T07:35:51.868 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: created 0.000000 2024-03-13T07:35:51.868 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: min_mon_release 0 (unknown) 2024-03-13T07:35:51.868 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: election_strategy: 1 2024-03-13T07:35:51.868 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:35:51.868 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:35:51.868 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:35:51.869 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.869 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.747+0000 7f71e0bf5780 1 Processor -- start 2024-03-13T07:35:51.869 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.747+0000 7f71e0bf5780 1 -- start start 2024-03-13T07:35:51.869 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.747+0000 7f71e0bf5780 10 monclient: init 2024-03-13T07:35:51.869 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.748+0000 7f71e0bf5780 10 monclient: _reopen_session rank -1 2024-03-13T07:35:51.869 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.748+0000 7f71e0bf5780 10 monclient: _add_conns ranks=[1,2,0] 2024-03-13T07:35:51.869 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.748+0000 7f71e0bf5780 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2c1702eb0 0x55a2c179a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:35:51.869 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.748+0000 7f71e0bf5780 10 monclient(hunting): picked mon.noname-b con 0x55a2c1702eb0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:35:51.869 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.748+0000 7f71e0bf5780 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2c17958a0 0x55a2c179a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:35:51.869 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.748+0000 7f71e0bf5780 10 monclient(hunting): picked mon.noname-c con 0x55a2c17958a0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:35:51.869 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.748+0000 7f71e0bf5780 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2c179ada0 0x55a2c16fea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:35:51.869 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.748+0000 7f71e0bf5780 10 monclient(hunting): picked mon.noname-a con 0x55a2c179ada0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:35:51.869 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.748+0000 7f71e0bf5780 10 monclient(hunting): start opening mon connection 2024-03-13T07:35:51.869 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.748+0000 7f71e0bf5780 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c1798e30 con 0x55a2c179ada0 2024-03-13T07:35:51.869 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.748+0000 7f71e0bf5780 10 monclient(hunting): start opening mon connection 2024-03-13T07:35:51.870 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.748+0000 7f71e0bf5780 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c1674c40 con 0x55a2c1702eb0 2024-03-13T07:35:51.870 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.748+0000 7f71e0bf5780 10 monclient(hunting): start opening mon connection 2024-03-13T07:35:51.870 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.748+0000 7f71e0bf5780 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2c1797f00 con 0x55a2c17958a0 2024-03-13T07:35:51.870 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.748+0000 7f71e0bf5780 10 monclient(hunting): _renew_subs 2024-03-13T07:35:51.870 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.748+0000 7f71e0bf5780 10 monclient(hunting): authenticate will time out at 1379.156516s 2024-03-13T07:35:51.873 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.873 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.873 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.873 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.873 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.873 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.873 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.873 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.873 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.873 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.873 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.873 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.873 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.874 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/40938/exe 2024-03-13T07:35:51.874 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.874 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.874 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.874 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.874 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.874 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.874 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.874 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.874 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.874 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.874 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.874 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.874 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.875 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.875 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.875 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.875 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.875 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.875 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.875 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_stat_pp 2024-03-13T07:35:51.875 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.875 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41086 2024-03-13T07:35:51.875 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_stat_pp on pid 41086' 2024-03-13T07:35:51.875 INFO:tasks.workunit.client.0.smithi132.stdout: test api_stat_pp on pid 41086 2024-03-13T07:35:51.877 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41086 2024-03-13T07:35:51.877 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.877 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.877 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.877 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-03-13T07:35:51.878 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.878 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.878 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.878 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.878 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.878 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.878 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.878 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.878 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.878 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.878 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/40922/exe 2024-03-13T07:35:51.878 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.879 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.753+0000 7f71ddef4640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2c17958a0 0x55a2c179a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:35:51.879 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.753+0000 7f71ddef4640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2c17958a0 0x55a2c179a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:57494/0 (socket says 172.21.15.132:57494) 2024-03-13T07:35:51.879 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.753+0000 7f71ddef4640 1 -- 172.21.15.132:0/342422605 learned_addr learned my addr 172.21.15.132:0/342422605 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:35:51.879 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.753+0000 7f71ddef4640 10 monclient(hunting): get_auth_request con 0x55a2c17958a0 auth_method 0 2024-03-13T07:35:51.879 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.753+0000 7f71ddef4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:35:51.879 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.753+0000 7f71ddef4640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:35:51.879 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.753+0000 7f71ddef4640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:35:51.879 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.755+0000 7f71ddef4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:35:51.879 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.755+0000 7f71ddef4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:35:51.879 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.755+0000 7f71ddef4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:35:51.879 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.756+0000 7f71de6f5640 1 --2- 172.21.15.132:0/342422605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2c1702eb0 0x55a2c179a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:35:51.879 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.757+0000 7f71de6f5640 10 monclient(hunting): get_auth_request con 0x55a2c1702eb0 auth_method 0 2024-03-13T07:35:51.879 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.757+0000 7f71de6f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:35:51.879 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.757+0000 7f71de6f5640 10 monclient(hunting): _init_auth method 2 2024-03-13T07:35:51.880 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.757+0000 7f71de6f5640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:35:51.880 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.758+0000 7f71de6f5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:35:51.880 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.758+0000 7f71de6f5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:35:51.880 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.758+0000 7f71de6f5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:35:51.880 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.759+0000 7f71ddef4640 10 monclient(hunting): handle_auth_done global_id 4364 payload 274 2024-03-13T07:35:51.880 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.759+0000 7f71ddef4640 1 -- 172.21.15.132:0/342422605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2c1702eb0 msgr2=0x55a2c179a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:35:51.880 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.759+0000 7f71ddef4640 1 --2- 172.21.15.132:0/342422605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2c1702eb0 0x55a2c179a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:35:51.880 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.759+0000 7f71ddef4640 1 -- 172.21.15.132:0/342422605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2c179ada0 msgr2=0x55a2c16fea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T07:35:51.888 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2024-03-13T07:35:51.888 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.888 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.888 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.888 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.888 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.888 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.888 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.888 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.888 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.889 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.889 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.889 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.889 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.889 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.889 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.889 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.889 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_pool.log 2024-03-13T07:35:51.889 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_pool: /' 2024-03-13T07:35:51.889 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.889 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.889 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.889 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_tier_pp: /' 2024-03-13T07:35:51.889 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.889 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.890 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.890 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.890 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.890 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.890 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.890 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.890 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.890 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.890 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.890 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.890 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.890 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.890 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.890 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.891 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.891 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.891 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.891 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.891 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.891 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.891 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2024-03-13T07:35:51.891 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.891 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.891 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.891 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.891 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.891 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.891 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.891 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.891 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.892 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.892 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.892 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-03-13T07:35:51.892 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.892 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.892 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.892 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_watch_notify 2024-03-13T07:35:51.892 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.892 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.892 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.892 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.892 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.892 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.892 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.892 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.893 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.893 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.893 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.893 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.893 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.893 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.893 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.893 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.893 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.893 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_watch_notify' 2024-03-13T07:35:51.893 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.893 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.893 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.893 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.893 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.894 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.894 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.894 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.894 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.894 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.894 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_watch_notify 2024-03-13T07:35:51.894 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.894 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.894 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.894 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.894 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.894 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.894 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.894 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.894 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.895 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.895 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.895 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.895 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.895 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.895 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.895 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.895 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.895 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.895 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.895 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.895 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.895 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.895 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.896 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.896 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.896 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.896 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_watch_notify 2024-03-13T07:35:51.896 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41133 2024-03-13T07:35:51.896 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_watch_notify on pid 41133' 2024-03-13T07:35:51.896 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.759+0000 7f71ddef4640 1 --2- 172.21.15.132:0/342422605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2c179adatest api_watch_notify on pid 41133 2024-03-13T07:35:51.903 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41133 2024-03-13T07:35:51.903 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.903 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.903 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.903 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.904 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.904 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.904 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.904 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.904 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.904 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.904 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.904 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.904 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.904 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.904 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.904 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.904 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.905 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.905 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.905 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.905 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.905 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.905 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.905 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.905 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.905 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.905 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.905 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.905 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.905 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.905 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.905 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.905 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.906 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.906 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.906 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.906 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.906 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.906 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.906 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.906 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.906 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.906 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.906 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41012/exe 2024-03-13T07:35:51.906 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.906 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.906 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.907 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.907 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.907 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.907 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.907 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.907 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.907 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.907 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.907 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.907 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.907 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.907 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.907 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.907 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.907 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.907 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.908 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.908 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.908 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.908 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.908 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.908 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.908 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.908 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.908 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.908 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.908 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.908 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.908 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.908 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.908 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.909 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.909 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.909 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.909 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_watch_notify_pp 2024-03-13T07:35:51.909 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_watch_notify_pp' 2024-03-13T07:35:51.909 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.909 INFO:tasks.workunit.client.0.smithi132.stdout:0 0x55a2c16fea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:35:51.909 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.759+0000 7f71ddef4640 10 monclient: _finish_hunting 0 2024-03-13T07:35:51.909 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.759+0000 7f71ddef4640 1 monclient: found mon.noname-c 2024-03-13T07:35:51.909 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.759+0000 7f71ddef4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:35:51.909 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.759+0000 7f71ddef4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:35:51.909 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.759+0000 7f71ddef4640 1 -- 172.21.15.132:0/342422605 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2c1799e00 con 0x55a2c17958a0 2024-03-13T07:35:51.910 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.768+0000 7f71ddef4640 1 --2- 172.21.15.132:0/342422605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2c17958a0 0x55a2c179a860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71d400a5e0 tx=0x7f71d4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af5528734253cea server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:35:51.910 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71dcef2640 1 -- 172.21.15.132:0/342422605 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4008a40 con 0x55a2c17958a0 2024-03-13T07:35:51.910 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71dcef2640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:35:51.910 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71dcef2640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:35:51.910 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71dcef2640 10 monclient: dump: 2024-03-13T07:35:51.910 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: epoch 1 2024-03-13T07:35:51.910 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:35:51.910 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:35:51.910 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:35:51.910 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: min_mon_release 19 (squid) 2024-03-13T07:35:51.910 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: election_strategy: 1 2024-03-13T07:35:51.910 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:35:51.910 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:35:51.910 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:35:51.911 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.911 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71dcef2640 10 monclient: _finish_auth 0 2024-03-13T07:35:51.911 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71dcef2640 10 monclient: _check_auth_tickets 2024-03-13T07:35:51.911 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71dcef2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:35:51.911 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71dcef2640 1 -- 172.21.15.132:0/342422605 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d4008c00 con 0x55a2c17958a0 2024-03-13T07:35:51.911 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71e0bf5780 5 monclient: authenticate success, global_id 4364 2024-03-13T07:35:51.911 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71e0bf5780 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T07:35:51.911 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71dcef2640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:35:51.911 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71dcef2640 1 -- 172.21.15.132:0/342422605 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4013440 con 0x55a2c17958a0 2024-03-13T07:35:51.911 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71dcef2640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:35:51.911 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71dcef2640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:35:51.911 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71dcef2640 10 monclient: dump: 2024-03-13T07:35:51.911 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: epoch 1 2024-03-13T07:35:51.911 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:35:51.911 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:35:51.912 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_snapshots: /' 2024-03-13T07:35:51.912 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-03-13T07:35:51.912 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-03-13T07:35:51.912 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:35:51.912 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: min_mon_release 19 (squid) 2024-03-13T07:35:51.913 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: election_strategy: 1 2024-03-13T07:35:51.913 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:35:51.913 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:35:51.913 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:35:51.913 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.913 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71e0bf5780 10 monclient: get_monmap_and_config success 2024-03-13T07:35:51.913 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.788+0000 7f71e0bf5780 4 set_mon_vals no callback set 2024-03-13T07:35:51.913 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.789+0000 7f71e0bf5780 10 monclient: shutdown 2024-03-13T07:35:51.913 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.789+0000 7f71e0bf5780 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:35:51.913 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.789+0000 7f71e0bf5780 1 -- 172.21.15.132:0/342422605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2c17958a0 msgr2=0x55a2c179a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:35:51.913 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.789+0000 7f71e0bf5780 1 --2- 172.21.15.132:0/342422605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2c17958a0 0x55a2c179a860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71d400a5e0 tx=0x7f71d4005f90 comp rx=0 tx=0).stop 2024-03-13T07:35:51.913 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.789+0000 7f71e0bf5780 1 -- 172.21.15.132:0/342422605 shutdown_connections 2024-03-13T07:35:51.913 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.789+0000 7f71e0bf5780 1 --2- 172.21.15.132:0/342422605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2c179ada0 0x55a2c16fea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:35:51.913 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.789+0000 7f71e0bf5780 1 --2- 172.21.15.132:0/342422605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2c17958a0 0x55a2c179a860 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:35:51.913 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.789+0000 7f71e0bf5780 1 --2- 172.21.15.132:0/342422605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2c1702eb0 0x55a2c179a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:35:51.914 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.789+0000 7f71e0bf5780 1 -- 172.21.15.132:0/342422605 >> 172.21.15.132:0/342422605 conn(0x55a2c16c0290 msgr2=0x55a2c16fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:35:51.914 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.791+0000 7f71e0bf5780 1 -- 172.21.15.132:0/342422605 shutdown_connections 2024-03-13T07:35:51.914 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.791+0000 7f71e0bf5780 1 -- 172.21.15.132:0/342422605 wait complete. 2024-03-13T07:35:51.914 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.791+0000 7f71e0bf5780 10 monclient: build_initial_monmap 2024-03-13T07:35:51.914 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.791+0000 7f71e0bf5780 10 monclient: monmap: 2024-03-13T07:35:51.914 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: epoch 0 2024-03-13T07:35:51.914 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: fsid 00000000-0000-0000-0000-000000000000 2024-03-13T07:35:51.914 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: last_changed 0.000000 2024-03-13T07:35:51.914 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: created 0.000000 2024-03-13T07:35:51.914 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: min_mon_release 0 (unknown) 2024-03-13T07:35:51.914 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: election_strategy: 1 2024-03-13T07:35:51.914 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T07:35:51.914 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T07:35:51.914 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T07:35:51.914 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.915 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.791+0000 7f71e0bf5780 1 librados: starting msgr at 2024-03-13T07:35:51.915 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.791+0000 7f71e0bf5780 1 librados: starting objecter 2024-03-13T07:35:51.915 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.791+0000 7f71e0bf5780 1 Processor -- start 2024-03-13T07:35:51.915 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.791+0000 7f71e0bf5780 1 -- start start 2024-03-13T07:35:51.915 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.791+0000 7f71e0bf5780 1 librados: setting wanted keys 2024-03-13T07:35:51.916 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.916 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.916 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.916 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.916 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.916 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.916 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.916 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.916 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.917 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.917 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.917 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.917 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.917 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.917 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.917 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.791+0000 7f71e0bf5780 1 librados: calling monclient init 2024-03-13T07:35:51.917 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.791+0000 7f71e0bf5780 10 monclient: init 2024-03-13T07:35:51.917 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.792+0000 7f71e0bf5780 10 monclient: _reopen_session rank -1 2024-03-13T07:35:51.917 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.792+0000 7f71e0bf5780 10 monclient: _add_conns ranks=[0,1,2] 2024-03-13T07:35:51.917 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.792+0000 7f71e0bf5780 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2c1702eb0 0x55a2c183af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:35:51.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.792+0000 7f71e0bf5780 10 monclient(hunting): picked mon.noname-a con 0x55a2c1702eb0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T07:35:51.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.792+0000 7f71e0bf5780 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2c17958a0 0x55a2c183db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:35:51.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.792+0000 7f71e0bf5780 10 monclient(hunting): picked mon.noname-b con 0x55a2c17958a0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T07:35:51.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.792+0000 7f71e0bf5780 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2c179ada0 0x55a2c16c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:35:51.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.792+0000 7f71e0bf5780 10 monclient(hunting): picked mon.noname-c con 0x55a2c179ada0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T07:35:51.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.792+0000 7f71e0bf5780 10 monclient(hunting): start opening mon connection 2024-03-13T07:35:51.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.792+0000 7f71e0bf5780 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c16fefd0 con 0x55a2c1702eb0 2024-03-13T07:35:51.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.792+0000 7f71e0bf5780 10 monclient(hunting): start opening mon connection 2024-03-13T07:35:51.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.792+0000 7f71e0bf5780 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c178a950 con 0x55a2c17958a0 2024-03-13T07:35:51.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.792+0000 7f71e0bf5780 10 monclient(hunting): start opening mon connection 2024-03-13T07:35:51.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.792+0000 7f71e0bf5780 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2c16ff150 con 0x55a2c179ada0 2024-03-13T07:35:51.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.792+0000 7f71e0bf5780 10 monclient(hunting): _renew_subs 2024-03-13T07:35:51.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.792+0000 7f71e0bf5780 10 monclient(hunting): authenticate will time out at 1379.200928s 2024-03-13T07:35:51.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.793+0000 7f71deef6640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2c179ada0 0x55a2c16c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:35:51.919 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.794+0000 7f71deef6640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2c179ada0 0x55a2c16c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:57508/0 (socket says 172.21.15.132:57508) 2024-03-13T07:35:51.919 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.794+0000 7f71deef6640 1 -- 172.21.15.132:0/2636667481 learned_addr learned my addr 172.21.15.132:0/2636667481 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T07:35:51.919 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.794+0000 7f71deef6640 10 monclient(hunting): get_auth_request con 0x55a2c179ada0 auth_method 0 2024-03-13T07:35:51.919 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.794+0000 7f71deef6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T07:35:51.921 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.921 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.921 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.922 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.922 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.922 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.922 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.922 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.922 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.922 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.922 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.922 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.922 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.922 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.922 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.922 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.922 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.923 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.923 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_watch_notify_pp 2024-03-13T07:35:51.923 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.923 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2024-03-13T07:35:51.923 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.923 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.923 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.923 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.923 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.923 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_watch_notify_pp 2024-03-13T07:35:51.923 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41182 2024-03-13T07:35:51.923 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_watch_notify_pp on pid 41182' 2024-03-13T07:35:51.924 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.794+0000 7f71deef6640 10 monclient(hunting): _initest api_watch_notify_pp on pid 41182 2024-03-13T07:35:51.942 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41182 2024-03-13T07:35:51.942 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.942 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.942 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_cmd 2024-03-13T07:35:51.942 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_cmd' 2024-03-13T07:35:51.942 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.942 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.942 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.942 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.942 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.943 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.943 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.943 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.943 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.943 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.943 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.943 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.943 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.943 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.943 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.943 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.943 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.943 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.944 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.944 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.944 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.944 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.944 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.944 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.944 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-03-13T07:35:51.944 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.944 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.944 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.944 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.944 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.944 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.944 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.944 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.945 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.945 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.945 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.945 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.945 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.945 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.945 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.945 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.945 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.945 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.945 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.945 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.945 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.945 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.945 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.946 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.946 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.946 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.946 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.946 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.946 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.946 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.946 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.946 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.946 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.946 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.946 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.946 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.946 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.947 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.947 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.947 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.947 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.947 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.947 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.947 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.947 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.947 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.947 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.947 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.947 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_cmd 2024-03-13T07:35:51.947 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.947 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.947 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.948 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.948 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.948 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.948 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.948 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.948 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.948 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.948 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.948 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.948 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.948 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.948 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.948 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.948 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.948 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.949 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.949 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.949 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.949 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.949 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.949 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.949 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.949 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.949 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.949 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.949 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.949 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.949 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.949 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.949 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.950 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.950 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.950 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.950 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.950 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.950 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.950 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.950 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.950 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.950 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.950 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.950 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41034/exe 2024-03-13T07:35:51.950 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.950 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.950 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.951 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.951 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.951 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.951 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.951 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.951 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.951 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.951 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.951 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.951 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.951 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.951 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.951 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.951 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.951 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.951 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.952 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.952 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.952 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.952 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.952 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.952 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.952 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.952 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.952 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.952 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.952 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.952 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.952 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.952 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.952 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.953 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.953 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.953 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.953 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.953 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.953 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.953 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.953 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.953 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.953 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.953 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.953 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.953 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.953 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.953 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.954 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.954 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.954 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.954 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.954 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.954 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.954 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.954 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.954 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.954 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.954 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.954 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.954 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.954 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.954 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.955 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.955 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_cmd 2024-03-13T07:35:51.955 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.955 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.955 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.955 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.955 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.955 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.955 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41228 2024-03-13T07:35:51.955 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.955 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.956 INFO:tasks.workunit.client.0.smithi132.stdout:test api_cmd on pid 41228 2024-03-13T07:35:51.962 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_cmd on pid 41228' 2024-03-13T07:35:51.962 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41228 2024-03-13T07:35:51.962 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.962 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.962 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.962 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2024-03-13T07:35:51.962 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-03-13T07:35:51.962 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-03-13T07:35:51.963 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_cmd_pp 2024-03-13T07:35:51.963 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-03-13T07:35:51.963 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_cmd_pp' 2024-03-13T07:35:51.963 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41054/exe 2024-03-13T07:35:51.963 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41086/exe 2024-03-13T07:35:51.963 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.963 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.963 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.963 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.963 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.963 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.963 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.963 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.963 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.963 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.965 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.965 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.965 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.965 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.965 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.965 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.965 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.965 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.965 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.965 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.965 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.965 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.965 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.965 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.965 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.966 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_stat.log 2024-03-13T07:35:51.966 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2024-03-13T07:35:51.966 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.966 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.966 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.966 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.966 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.966 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.966 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.966 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.966 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.966 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.966 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_cmd_pp 2024-03-13T07:35:51.966 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_stat: /' 2024-03-13T07:35:51.966 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:51.967 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.967 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.967 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:51.967 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.967 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:51.967 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.967 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.967 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.967 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:51.967 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.967 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.967 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.967 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:51.967 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.967 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:51.968 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.968 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:51.968 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.968 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:51.968 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.968 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:51.968 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:51.968 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:51.968 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.968 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:51.968 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.968 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:51.968 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.968 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.969 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.969 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.969 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.969 INFO:tasks.workunit.client.0.smithi132.stdout:test api_cmd_pp on pid 41255 2024-03-13T07:35:51.977 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.977 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.977 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.977 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.977 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.977 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.977 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.977 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.977 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.977 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.977 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.977 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.977 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.978 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.978 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.978 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.978 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.978 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.978 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:51.978 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:51.978 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:51.978 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.978 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_cmd_pp 2024-03-13T07:35:51.978 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:51.978 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.978 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:51.978 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:51.979 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.979 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:51.979 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.979 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:51.979 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41255 2024-03-13T07:35:51.979 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_cmd_pp on pid 41255' 2024-03-13T07:35:51.979 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41255 2024-03-13T07:35:51.979 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.979 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.979 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.979 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:51.979 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.979 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:51.979 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:51.979 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.980 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.980 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-03-13T07:35:51.980 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.980 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.980 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.980 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:51.980 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_service 2024-03-13T07:35:51.980 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_service' 2024-03-13T07:35:51.980 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.980 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:51.980 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.980 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:51.980 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:51.980 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:51.981 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.981 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_service 2024-03-13T07:35:51.981 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.981 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.981 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.981 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.981 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.981 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.981 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.981 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.981 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.981 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.981 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.981 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.981 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.982 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.982 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.982 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.982 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.982 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.982 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.982 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.982 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.982 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.982 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.982 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.982 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.982 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.983 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.983 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.983 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.983 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.983 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.983 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.983 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.983 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.983 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.983 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.983 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:51.983 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.983 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.984 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.984 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.984 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.984 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.984 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.984 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.984 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.984 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.984 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.984 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.984 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.984 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:51.984 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:51.984 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:51.985 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:51.985 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:51.985 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.985 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:51.985 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:51.985 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:51.985 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:51.985 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:51.985 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.985 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:51.985 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.985 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:51.985 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:51.986 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.986 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:51.986 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.986 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:51.986 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:51.986 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2024-03-13T07:35:51.986 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_service 2024-03-13T07:35:51.986 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41272 2024-03-13T07:35:51.986 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_service on pid 41272' 2024-03-13T07:35:51.987 INFO:tasks.workunit.client.0.smithi132.stdout:test api_service on pid 41272 2024-03-13T07:35:51.991 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41272 2024-03-13T07:35:51.991 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:51.992 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:51.992 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_stat_pp: /' 2024-03-13T07:35:51.992 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-03-13T07:35:51.992 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41133/exe 2024-03-13T07:35:51.992 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:51.992 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:51.992 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:51.992 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:51.992 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:51.992 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:51.992 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:51.992 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:51.992 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:51.992 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:51.993 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:51.993 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-03-13T07:35:51.993 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:51.993 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:51.993 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_service_pp 2024-03-13T07:35:51.993 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:51.993 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:51.993 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.993 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_service_pp' 2024-03-13T07:35:51.993 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:51.993 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:51.993 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:51.993 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:51.993 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_service_pp 2024-03-13T07:35:51.993 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:51.995 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:51.995 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:51.995 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:51.995 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:51.995 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:51.995 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:51.995 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:51.995 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.995 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:51.995 INFO:tasks.workunit.client.0.smithi132.stdout:t_auth method 2 2024-03-13T07:35:51.995 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.794+0000 7f71deef6640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T07:35:51.996 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.794+0000 7f71deef6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T07:35:51.996 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.794+0000 7f71deef6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T07:35:51.996 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.794+0000 7f71deef6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T07:35:51.996 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.799+0000 7f71deef6640 10 monclient(hunting): handle_auth_done global_id 4373 payload 824 2024-03-13T07:35:51.996 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.799+0000 7f71deef6640 1 -- 172.21.15.132:0/2636667481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2c17958a0 msgr2=0x55a2c183db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:35:51.996 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.799+0000 7f71deef6640 1 --2- 172.21.15.132:0/2636667481 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a2c17958a0 0x55a2c183db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:35:51.996 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.799+0000 7f71deef6640 1 -- 172.21.15.132:0/2636667481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2c1702eb0 msgr2=0x55a2c183af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:35:51.996 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.799+0000 7f71deef6640 1 --2- 172.21.15.132:0/2636667481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2c1702eb0 0x55a2c183af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:35:51.996 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.799+0000 7f71deef6640 10 monclient: _finish_hunting 0 2024-03-13T07:35:51.996 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.799+0000 7f71deef6640 1 monclient: found mon.noname-c 2024-03-13T07:35:51.996 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.799+0000 7f71deef6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:35:51.996 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.799+0000 7f71deef6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T07:35:51.996 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.799+0000 7f71deef6640 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2c183b4c0 con 0x55a2c179ada0 2024-03-13T07:35:51.996 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.801+0000 7f71deef6640 1 --2- 172.21.15.132:0/2636667481 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2c179ada0 0x55a2c16c7580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005e80 tx=0x7f71d8008430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:35:51.996 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.801+0000 7f71cf7fe640 10 client.?.objecter ms_handle_connect 0x55a2c179ada0 2024-03-13T07:35:51.997 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.801+0000 7f71cf7fe640 10 client.?.objecter resend_mon_ops 2024-03-13T07:35:51.997 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8010440 con 0x55a2c179ada0 2024-03-13T07:35:51.997 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:35:51.997 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71cf7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T07:35:51.997 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71cf7fe640 10 monclient: dump: 2024-03-13T07:35:51.997 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: epoch 1 2024-03-13T07:35:51.997 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:35:51.997 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:35:51.997 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:35:51.997 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: min_mon_release 19 (squid) 2024-03-13T07:35:51.997 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: election_strategy: 1 2024-03-13T07:35:51.997 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:35:52.002 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.002 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.002 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.002 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.002 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.002 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.003 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.003 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.003 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.003 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.003 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.003 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.003 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.003 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.003 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.003 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.003 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.003 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.003 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.003 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.003 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.004 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.004 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.004 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.004 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-03-13T07:35:52.004 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.004 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.004 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.004 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.004 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.004 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.004 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.004 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.004 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.004 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.005 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.005 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.005 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.005 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_watch_notify: /' 2024-03-13T07:35:52.005 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.005 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.005 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.005 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.005 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.005 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.005 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.005 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.005 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.005 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.005 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41182/exe 2024-03-13T07:35:52.006 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2024-03-13T07:35:52.006 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.006 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_service_pp 2024-03-13T07:35:52.006 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41304 2024-03-13T07:35:52.006 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_service_pp on pid 41304' 2024-03-13T07:35:52.006 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 1: [v2:172.21.15.195:3300/test api_service_pp on pid 41304 2024-03-13T07:35:52.009 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41304 2024-03-13T07:35:52.009 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:52.009 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.010 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.010 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.010 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-03-13T07:35:52.010 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.010 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.010 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.010 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.010 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.010 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.010 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.010 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.010 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.010 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.010 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.011 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.011 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.011 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.011 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.011 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2024-03-13T07:35:52.011 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_c_write_operations 2024-03-13T07:35:52.011 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_c_write_operations' 2024-03-13T07:35:52.011 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_c_write_operations 2024-03-13T07:35:52.011 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.011 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_c_write_operations 2024-03-13T07:35:52.011 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-03-13T07:35:52.011 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41323 2024-03-13T07:35:52.011 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.011 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.012 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.012 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.012 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.012 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.012 INFO:tasks.workunit.client.0.smithi132.stdout:test api_c_write_operations on pid 41323 2024-03-13T07:35:52.019 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_c_write_operations on pid 41323' 2024-03-13T07:35:52.020 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41323 2024-03-13T07:35:52.020 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:52.020 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.020 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.020 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.020 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.020 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.020 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.020 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.020 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_c_read_operations 2024-03-13T07:35:52.020 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.020 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.021 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.021 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.021 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.021 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.021 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.021 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.021 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.021 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_c_read_operations' 2024-03-13T07:35:52.021 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.021 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.021 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.021 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.021 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.021 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.022 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.022 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.022 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.022 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-03-13T07:35:52.022 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.022 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.022 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.022 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.022 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.022 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.022 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.022 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.022 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.022 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.023 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.023 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.023 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.023 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.023 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.023 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_c_read_operations 2024-03-13T07:35:52.023 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.023 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.023 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.023 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.023 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.023 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.023 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.023 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.023 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.024 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.024 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.024 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.024 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.024 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.024 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.024 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.024 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.024 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.024 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.024 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.024 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.024 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.024 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-03-13T07:35:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41228/exe 2024-03-13T07:35:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.026 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.026 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.026 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.026 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.026 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.026 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.026 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.026 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.026 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.026 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.026 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.026 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.026 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.027 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.027 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.027 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.027 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.027 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.027 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.027 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.027 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.027 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_cmd.log 2024-03-13T07:35:52.027 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_cmd: /' 2024-03-13T07:35:52.027 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_c_read_operations 2024-03-13T07:35:52.027 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41366 2024-03-13T07:35:52.027 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_c_read_operations on pid 41366' 2024-03-13T07:35:52.028 INFO:tasks.workunit.client.0.smithi132.stdout:test api_c_read_operations on pid 41366 2024-03-13T07:35:52.043 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41366 2024-03-13T07:35:52.043 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:52.043 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.043 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2024-03-13T07:35:52.043 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.043 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.043 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.044 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.044 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-03-13T07:35:52.044 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.044 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.044 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.044 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.044 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.044 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.044 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.044 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.044 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.044 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.044 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.044 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.044 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.045 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.045 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.045 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.045 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.045 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.045 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.045 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.045 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.045 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.045 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.045 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.045 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.045 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.045 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.045 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.046 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.046 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.046 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.046 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.046 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.046 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.046 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.046 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.046 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.046 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.046 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.046 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.046 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.046 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.047 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.047 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.047 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.047 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.047 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.047 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.047 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.047 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.047 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.047 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.047 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.047 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.047 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.047 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.048 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.048 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.048 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.048 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.048 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.048 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.048 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.048 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.048 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.048 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.048 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.048 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.048 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.048 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.048 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.049 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.049 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.049 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.049 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.049 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.049 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.049 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s api_cls_remote_reads 2024-03-13T07:35:52.049 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41255/exe 2024-03-13T07:35:52.049 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' api_cls_remote_reads' 2024-03-13T07:35:52.049 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.049 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.049 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.049 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.049 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.049 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.049 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.050 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.050 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.050 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.050 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.050 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.050 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.050 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.050 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.050 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.050 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.050 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.050 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.050 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.050 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo api_cls_remote_reads 2024-03-13T07:35:52.050 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.051 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.051 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.051 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.051 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.051 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.051 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.051 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.051 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.051 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.051 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.051 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.051 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.051 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.051 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.052 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.052 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.052 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.052 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.052 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.052 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.052 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.052 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.052 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.052 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.052 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.052 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.052 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.052 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.052 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-03-13T07:35:52.053 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.053 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.053 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.053 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.053 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.053 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.053 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.053 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.053 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.053 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.053 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.053 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.053 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.053 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.053 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.053 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.054 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.054 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.054 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.054 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.054 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.054 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.054 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.054 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.054 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.054 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.054 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.054 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.054 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.054 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.055 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.055 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.055 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.055 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.055 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.055 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.055 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.055 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.055 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.055 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.055 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.055 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.055 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.055 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.055 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.055 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.056 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.056 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.056 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2024-03-13T07:35:52.056 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.056 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.056 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.056 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.056 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.056 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.056 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.056 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=api_cls_remote_reads 2024-03-13T07:35:52.056 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41400 2024-03-13T07:35:52.057 INFO:tasks.workunit.client.0.smithi132.stdout:test api_cls_remote_reads on pid 41400 2024-03-13T07:35:52.062 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test api_cls_remote_reads on pid 41400' 2024-03-13T07:35:52.062 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41400 2024-03-13T07:35:52.063 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:52.063 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.063 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.063 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.063 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.063 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.063 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.063 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.063 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.063 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.063 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.063 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.063 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.063 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.063 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.064 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.064 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.064 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.064 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.064 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.064 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.064 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.064 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.064 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-03-13T07:35:52.064 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.064 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.064 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s list_parallel 2024-03-13T07:35:52.064 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.064 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.064 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.065 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.065 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.065 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.065 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.065 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' list_parallel' 2024-03-13T07:35:52.065 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.065 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.065 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.065 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.065 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.065 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.065 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.065 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.065 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.065 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.066 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo list_parallel 2024-03-13T07:35:52.066 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.066 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.066 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.066 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.066 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.066 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.066 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.066 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.066 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.066 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.066 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.066 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.066 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.066 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.067 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.067 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.067 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.067 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.067 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=list_parallel 2024-03-13T07:35:52.067 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.067 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.067 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41418 2024-03-13T07:35:52.067 INFO:tasks.workunit.client.0.smithi132.stdout:test list_parallel on pid 41418 2024-03-13T07:35:52.074 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test list_parallel on pid 41418' 2024-03-13T07:35:52.074 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41418 2024-03-13T07:35:52.074 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:52.074 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.074 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.074 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.075 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.075 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.075 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s open_pools_parallel 2024-03-13T07:35:52.075 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.075 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.075 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.075 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.075 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' open_pools_parallel' 2024-03-13T07:35:52.075 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.075 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.075 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.075 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.075 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.075 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.075 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41272/exe 2024-03-13T07:35:52.076 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.076 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.076 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.076 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.076 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.076 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.076 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.076 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.076 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.076 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.076 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.076 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.076 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo open_pools_parallel 2024-03-13T07:35:52.076 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.076 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.077 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-03-13T07:35:52.077 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.077 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.077 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.077 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.077 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2024-03-13T07:35:52.077 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.077 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.077 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.077 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.077 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.077 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.077 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.077 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.077 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.077 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.078 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.078 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.078 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.078 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.078 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.078 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.078 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.078 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.078 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.078 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.078 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.078 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.078 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.078 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.078 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.079 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.079 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.079 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.079 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.079 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.079 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.079 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.079 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.079 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.079 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.079 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=open_pools_parallel 2024-03-13T07:35:52.079 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.079 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41446 2024-03-13T07:35:52.080 INFO:tasks.workunit.client.0.smithi132.stdout:test open_pools_parallel on pid 41446 2024-03-13T07:35:52.081 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test open_pools_parallel on pid 41446' 2024-03-13T07:35:52.081 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41446 2024-03-13T07:35:52.081 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-13T07:35:52.081 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.082 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.082 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.082 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.082 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.082 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.082 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.082 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.082 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.082 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.082 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.082 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-03-13T07:35:52.082 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_service: /' 2024-03-13T07:35:52.083 INFO:tasks.workunit.client.0.smithi132.stdout:0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:35:52.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:35:52.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:52.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71cf7fe640 10 monclient: _finish_auth 0 2024-03-13T07:35:52.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71cf7fe640 10 monclient: _check_auth_tickets 2024-03-13T07:35:52.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:35:52.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d8015070 con 0x55a2c179ada0 2024-03-13T07:35:52.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71cf7fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T07:35:52.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8010860 con 0x55a2c179ada0 2024-03-13T07:35:52.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T07:35:52.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71cf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T07:35:52.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71cf7fe640 10 monclient: dump: 2024-03-13T07:35:52.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: epoch 1 2024-03-13T07:35:52.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:35:52.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T07:35:52.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: created 2024-03-13T07:33:05.110387+0000 2024-03-13T07:35:52.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: min_mon_release 19 (squid) 2024-03-13T07:35:52.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: election_strategy: 1 2024-03-13T07:35:52.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T07:35:52.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T07:35:52.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T07:35:52.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:52.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71e0bf5780 5 monclient: authenticate success, global_id 4373 2024-03-13T07:35:52.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71e0bf5780 10 monclient: _renew_subs 2024-03-13T07:35:52.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71e0bf5780 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:35:52.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a2c183e090 con 0x55a2c179ada0 2024-03-13T07:35:52.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71e0bf5780 10 client.4373.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:35:52.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71e0bf5780 10 monclient: _renew_subs 2024-03-13T07:35:52.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71e0bf5780 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:35:52.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2c183a610 con 0x55a2c179ada0 2024-03-13T07:35:52.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71e0bf5780 1 librados: init done 2024-03-13T07:35:52.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71e0bf5780 10 librados: wait_for_osdmap waiting 2024-03-13T07:35:52.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.845+0000 7f71dcef2640 4 set_mon_vals no callback set 2024-03-13T07:35:52.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.910+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f71d8010a00 con 0x55a2c179ada0 2024-03-13T07:35:52.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.910+0000 7f71cf7fe640 1 --2- 172.21.15.132:0/2636667481 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f71c006b7d0 0x7f71c006dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:35:52.098 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_service.log 2024-03-13T07:35:52.098 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.098 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.098 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.098 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.098 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.098 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.099 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.099 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.099 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.099 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.099 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.099 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.099 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.099 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.099 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.099 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.099 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.099 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.099 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.100 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.100 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.100 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.100 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.100 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.100 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.100 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.100 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.100 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.100 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.100 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.100 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.100 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.100 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.100 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.101 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.101 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.101 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.101 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.101 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.101 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.101 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41304/exe 2024-03-13T07:35:52.101 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-03-13T07:35:52.101 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.101 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.101 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.101 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.101 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.102 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.102 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.102 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.102 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.102 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.102 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s delete_pools_parallel 2024-03-13T07:35:52.102 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.102 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.102 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' delete_pools_parallel' 2024-03-13T07:35:52.102 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.102 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.102 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.102 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.102 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.103 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.103 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.103 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.103 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.103 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.103 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.103 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.103 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.103 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.103 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.103 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.103 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.103 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.104 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.104 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.104 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.104 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.104 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.104 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.104 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.104 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.104 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.104 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.104 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.104 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2024-03-13T07:35:52.105 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.105 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.105 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.105 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.105 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.105 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.105 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.105 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.105 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.105 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.105 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.105 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.105 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.105 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41366/exe 2024-03-13T07:35:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.108 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.108 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.108 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.108 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.108 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.108 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.108 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.108 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.108 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.108 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.108 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.108 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.108 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.108 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.108 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.108 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.109 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.109 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.109 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.109 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.109 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.109 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.109 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.109 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.109 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.109 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.109 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo delete_pools_parallel 2024-03-13T07:35:52.109 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=delete_pools_parallel 2024-03-13T07:35:52.109 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41508 2024-03-13T07:35:52.109 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test delete_pools_parallel on pid 41508' 2024-03-13T07:35:52.110 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.913+0000 7f71de6f5640 1 --2- 172.21.15.132:test delete_pools_parallel on pid 41508 2024-03-13T07:35:52.120 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41508 2024-03-13T07:35:52.120 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-13T07:35:52.120 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.120 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s cls 2024-03-13T07:35:52.120 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' cls' 2024-03-13T07:35:52.120 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-03-13T07:35:52.121 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2024-03-13T07:35:52.121 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.121 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.121 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.121 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.121 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-03-13T07:35:52.121 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.121 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.121 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.121 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.121 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.121 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.121 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.121 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.121 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.122 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.122 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.122 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.122 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.122 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo cls 2024-03-13T07:35:52.122 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_service_pp: /' 2024-03-13T07:35:52.122 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.122 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-03-13T07:35:52.122 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-03-13T07:35:52.122 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.122 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.122 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.122 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.122 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.122 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41400/exe 2024-03-13T07:35:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41323/exe 2024-03-13T07:35:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-03-13T07:35:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-03-13T07:35:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=cls 2024-03-13T07:35:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41537 2024-03-13T07:35:52.129 INFO:tasks.workunit.client.0.smithi132.stdout:test cls on pid 41537 2024-03-13T07:35:52.143 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test cls on pid 41537' 2024-03-13T07:35:52.143 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41537 2024-03-13T07:35:52.144 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-13T07:35:52.144 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.144 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2024-03-13T07:35:52.144 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-03-13T07:35:52.144 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2024-03-13T07:35:52.144 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s cmd 2024-03-13T07:35:52.144 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.144 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.144 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.144 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.144 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.144 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.144 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-03-13T07:35:52.144 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.144 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.145 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.145 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-03-13T07:35:52.145 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.145 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.145 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.145 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.145 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.145 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.145 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.145 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.145 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.145 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.146 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.146 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.146 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.146 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.146 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.146 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.146 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.146 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.146 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.146 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.146 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.146 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.147 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.147 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.147 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.147 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.147 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.147 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.147 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.147 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.147 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.147 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.147 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.147 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.147 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.147 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.148 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.148 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.148 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.148 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.148 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.148 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.148 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.148 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.148 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.148 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.148 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.148 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.148 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.148 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.149 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' cmd' 2024-03-13T07:35:52.149 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.149 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.149 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.149 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.149 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.149 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.149 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.149 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.149 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.149 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.149 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.149 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.149 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.149 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.150 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.150 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.150 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.150 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.150 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.150 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.150 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.150 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.150 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.150 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.150 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.150 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.150 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.150 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo cmd 2024-03-13T07:35:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.153 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.153 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.153 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.153 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.153 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.153 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.153 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.153 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.153 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.153 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.153 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.153 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.153 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.153 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.153 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.154 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.154 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.154 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.154 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=cmd 2024-03-13T07:35:52.154 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41582 2024-03-13T07:35:52.154 INFO:tasks.workunit.client.0.smithi132.stdout:test cmd on pid 41582 2024-03-13T07:35:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test cmd on pid 41582' 2024-03-13T07:35:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41582 2024-03-13T07:35:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-13T07:35:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-03-13T07:35:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s handler_error 2024-03-13T07:35:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' handler_error' 2024-03-13T07:35:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.177 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.177 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.177 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.177 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.177 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.177 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.177 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.177 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.177 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.177 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.177 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.177 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.178 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.178 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.178 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.178 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.178 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.178 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.178 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.178 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.178 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.178 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.178 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.178 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.178 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.179 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.179 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.179 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.179 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.179 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.179 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.179 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.179 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.179 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.179 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.179 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.179 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.179 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.180 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.180 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.180 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.180 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.180 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.180 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.180 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.180 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.180 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.180 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.180 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.180 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.180 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.181 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.181 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.181 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.181 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.181 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.181 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.181 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.181 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.181 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.181 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.181 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.181 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.181 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.181 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.182 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.182 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.182 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.182 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.182 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.182 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.182 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.182 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo handler_error 2024-03-13T07:35:52.182 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.182 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.182 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.182 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.182 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.182 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.183 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.183 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.183 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.183 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.183 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.183 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.183 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.183 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.183 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.183 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.183 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.183 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.183 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.184 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.184 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.184 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.184 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.184 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.184 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.184 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.184 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.184 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.184 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.184 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.184 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.184 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.185 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.185 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.185 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.185 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.185 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.185 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.185 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.185 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.185 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.185 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.185 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.185 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.185 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=handler_error 2024-03-13T07:35:52.185 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41619 2024-03-13T07:35:52.186 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test handler_error on pid 41619' 2024-03-13T07:35:52.186 INFO:tasks.workunit.client.0.smithi132.stdout:test handler_error on pid 41619 2024-03-13T07:35:52.198 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41619 2024-03-13T07:35:52.198 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-13T07:35:52.199 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.199 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-03-13T07:35:52.199 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.199 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.199 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.199 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.199 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.199 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.199 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.199 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.199 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.199 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41537/exe 2024-03-13T07:35:52.199 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.199 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.199 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.199 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.200 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.200 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.200 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.200 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.200 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.200 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.200 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.200 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.200 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.200 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.200 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.200 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.200 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.200 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.200 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.201 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s io 2024-03-13T07:35:52.201 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.201 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' io' 2024-03-13T07:35:52.201 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.201 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.201 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.201 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.201 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.201 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.201 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.201 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.201 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.201 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.201 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.201 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.202 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.202 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.202 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.202 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.202 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.202 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.202 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.202 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.202 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.202 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.202 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.202 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.202 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.202 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.202 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.203 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.203 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.203 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.203 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.203 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.203 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.203 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.203 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_neorados_cls 2024-03-13T07:35:52.203 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.203 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.203 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.203 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.203 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.203 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.203 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.204 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.204 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.204 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ cls: /' 2024-03-13T07:35:52.204 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_neorados_cls.log 2024-03-13T07:35:52.204 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo io 2024-03-13T07:35:52.204 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.204 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.204 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.204 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.204 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.204 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.204 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.204 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.204 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.204 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.204 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.205 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.205 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.205 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.205 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.205 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.205 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.205 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.205 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.205 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.205 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.205 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.205 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41418/exe 2024-03-13T07:35:52.205 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41508/exe 2024-03-13T07:35:52.205 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.205 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41446/exe 2024-03-13T07:35:52.206 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.206 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.206 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.206 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.206 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.206 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.206 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.206 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.206 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.206 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.206 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.206 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.206 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.206 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.207 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.207 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.207 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.207 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.207 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.207 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.207 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.207 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.207 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.207 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.207 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.207 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.207 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.207 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.207 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ list_parallel: /' 2024-03-13T07:35:52.208 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.208 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.208 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.208 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.208 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-03-13T07:35:52.208 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2024-03-13T07:35:52.208 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2024-03-13T07:35:52.208 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-03-13T07:35:52.208 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=io 2024-03-13T07:35:52.208 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41678 2024-03-13T07:35:52.208 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test io on pid 41678' 2024-03-13T07:35:52.209 INFO:tasks.workunit.client.0.smithi132.stdout:test io on pid 41678 2024-03-13T07:35:52.218 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41678 2024-03-13T07:35:52.218 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-13T07:35:52.218 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.218 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.218 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.218 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.218 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.218 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.218 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.218 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.218 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.218 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.219 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.219 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.219 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.219 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.219 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.219 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.219 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.219 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.219 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.219 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.219 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-03-13T07:35:52.219 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.219 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-03-13T07:35:52.219 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-03-13T07:35:52.219 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s ec_io 2024-03-13T07:35:52.220 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' ec_io' 2024-03-13T07:35:52.220 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.220 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.220 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.220 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.220 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.220 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.220 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.220 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2024-03-13T07:35:52.220 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.220 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.220 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.220 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.220 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_rados_list_parallel.log 2024-03-13T07:35:52.221 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.221 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.221 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.221 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.221 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.221 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.221 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.221 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.221 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.221 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.221 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.221 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.221 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.222 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.222 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.222 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.222 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.222 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.222 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.222 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.222 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.222 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.222 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.222 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.222 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.223 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.223 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.223 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.223 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.223 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.223 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.223 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo ec_io 2024-03-13T07:35:52.223 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.223 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.223 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.223 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.223 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.223 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.223 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.223 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.224 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.224 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.224 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.224 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.224 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.224 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.224 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.224 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.224 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.224 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.224 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.224 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=ec_io 2024-03-13T07:35:52.224 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.224 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.225 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.225 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.225 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41712 2024-03-13T07:35:52.225 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test ec_io on pid 41712' 2024-03-13T07:35:52.225 INFO:tasks.workunit.client.0.smithi132.stdout:test ec_io on pid 41712 2024-03-13T07:35:52.241 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41712 2024-03-13T07:35:52.241 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-13T07:35:52.241 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.242 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.242 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.242 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.242 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.242 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.242 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.242 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.242 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.242 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-03-13T07:35:52.242 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.242 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.242 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.242 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.243 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.243 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.243 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.243 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.243 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.243 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.243 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.243 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.243 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.243 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.243 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.243 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.243 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.243 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.244 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.244 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.244 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.244 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.244 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.244 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.244 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.244 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.244 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.244 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.244 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.244 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.244 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.244 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.245 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.245 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.245 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.245 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.245 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.245 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.245 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.245 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.245 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.245 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.245 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.245 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.245 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.246 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.246 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.246 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.246 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.246 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.246 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.246 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.246 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.246 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.246 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.246 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.246 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.246 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s list 2024-03-13T07:35:52.246 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' list' 2024-03-13T07:35:52.247 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.247 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.247 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.247 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.247 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.247 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.247 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.247 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.247 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.247 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.247 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.247 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.247 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.247 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.247 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.248 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.248 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.248 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.248 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.248 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.248 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.248 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.248 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.248 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.248 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.248 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.248 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.248 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41619/exe 2024-03-13T07:35:52.248 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.248 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.249 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.249 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.249 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.249 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.249 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.249 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.249 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.249 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.249 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.249 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.249 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.249 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.249 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.249 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.250 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.250 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.250 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.250 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.250 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.250 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.250 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.250 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.250 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.250 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.250 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.251 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.251 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.251 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.251 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.251 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.251 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.251 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.251 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.251 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.251 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.251 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.251 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.251 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.252 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.252 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.252 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.252 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.252 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.252 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.252 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.252 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.252 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.252 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.252 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.252 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.252 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.252 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.253 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.253 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.253 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.253 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.253 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_neorados_handler_error 2024-03-13T07:35:52.253 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.253 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.253 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.253 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.253 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.253 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.253 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.253 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.253 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.253 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.254 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.254 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ handler_error: /' 2024-03-13T07:35:52.254 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.254 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo list 2024-03-13T07:35:52.254 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_neorados_handler_error.log 2024-03-13T07:35:52.254 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.254 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41582/exe 2024-03-13T07:35:52.254 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.254 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.254 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.254 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.254 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.254 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.254 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.254 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.255 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.255 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.255 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.255 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.255 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.255 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.255 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.255 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.255 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.255 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_neorados_cmd 2024-03-13T07:35:52.255 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=list 2024-03-13T07:35:52.255 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41770 2024-03-13T07:35:52.255 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test list on pid 41770' 2024-03-13T07:35:52.256 INFO:tasks.workunit.client.0.smithi132.stdout:test list on pid 41770 2024-03-13T07:35:52.263 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41770 2024-03-13T07:35:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-13T07:35:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s ec_list 2024-03-13T07:35:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' ec_list' 2024-03-13T07:35:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.265 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.265 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo ec_list 2024-03-13T07:35:52.265 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-03-13T07:35:52.265 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_neorados_cmd.log 2024-03-13T07:35:52.265 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.265 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.265 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.265 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.265 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.265 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.265 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.265 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.265 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.265 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.265 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.266 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.266 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.266 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.266 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.266 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.266 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.266 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.266 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.266 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.266 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.266 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.266 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.266 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.268 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.268 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.268 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.268 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.268 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.268 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.268 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.268 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.268 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.268 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.268 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.268 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.268 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.269 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.269 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.269 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.269 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.269 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.269 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.269 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.269 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.269 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.269 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.269 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.270 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.270 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.270 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=ec_list 2024-03-13T07:35:52.270 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41790 2024-03-13T07:35:52.270 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test ec_list on pid 41790' 2024-03-13T07:35:52.270 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41678/exe 2024-03-13T07:35:52.270 INFO:tasks.workunit.client.0.smithi132.stdout:test ec_list on pid 41790 2024-03-13T07:35:52.277 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41790 2024-03-13T07:35:52.277 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-13T07:35:52.277 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.277 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-03-13T07:35:52.277 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.277 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.277 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.277 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.277 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.277 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.277 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.278 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.278 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.278 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.278 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.278 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_neorados_io 2024-03-13T07:35:52.278 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ cmd: /' 2024-03-13T07:35:52.278 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.278 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.278 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.278 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s misc 2024-03-13T07:35:52.278 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.278 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.279 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.279 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.279 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' misc' 2024-03-13T07:35:52.279 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.279 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.279 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.279 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.279 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.279 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.279 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.279 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.280 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.280 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.280 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.280 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.280 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.280 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.280 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.280 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.280 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.280 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.280 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.280 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.280 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.280 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.280 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.281 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.281 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.281 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo misc 2024-03-13T07:35:52.281 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.281 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ io: /' 2024-03-13T07:35:52.281 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.281 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.281 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.281 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.281 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.281 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.281 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.281 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.281 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.282 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.282 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.282 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=misc 2024-03-13T07:35:52.282 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.282 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.282 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41832 2024-03-13T07:35:52.282 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.282 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test misc on pid 41832' 2024-03-13T07:35:52.282 INFO:tasks.workunit.client.0.smithi132.stdout:test misc on pid 41832 2024-03-13T07:35:52.292 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41832 2024-03-13T07:35:52.292 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-13T07:35:52.292 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.292 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.292 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.292 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.292 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.292 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-03-13T07:35:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41712/exe 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s pool 2024-03-13T07:35:52.296 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.297 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.297 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.297 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.297 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' pool' 2024-03-13T07:35:52.297 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.297 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.297 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.297 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.297 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.297 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.297 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.297 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.297 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.297 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.297 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.297 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.298 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.298 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.298 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.298 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.298 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.298 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.298 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.298 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.298 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.298 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.298 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo pool 2024-03-13T07:35:52.298 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.299 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.299 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.299 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.299 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.299 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.299 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.299 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.299 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.299 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.299 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.299 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_neorados_ec_io 2024-03-13T07:35:52.300 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.300 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_neorados_ec_io.log 2024-03-13T07:35:52.300 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ ec_io: /' 2024-03-13T07:35:52.300 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=pool 2024-03-13T07:35:52.300 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41868 2024-03-13T07:35:52.300 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test pool on pid 41868' 2024-03-13T07:35:52.300 INFO:tasks.workunit.client.0.smithi132.stdout:test pool on pid 41868 2024-03-13T07:35:52.306 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41868 2024-03-13T07:35:52.306 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-13T07:35:52.306 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.307 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_neorados_io.log 2024-03-13T07:35:52.307 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s read_operations 2024-03-13T07:35:52.307 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' read_operations' 2024-03-13T07:35:52.307 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-03-13T07:35:52.307 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.307 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.307 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.307 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.307 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.307 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.307 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.307 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.307 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.307 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.308 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.308 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.308 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.308 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.308 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.308 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.308 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.308 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.308 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.308 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo read_operations 2024-03-13T07:35:52.308 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.308 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.308 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.308 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.308 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.309 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.309 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.309 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.309 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.309 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.309 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.309 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.309 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.309 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.309 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.309 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.309 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.309 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.309 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.309 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.310 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.310 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.310 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.310 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.310 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.310 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.310 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.310 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.310 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.310 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.310 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.310 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.310 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.310 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.311 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.311 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.311 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.311 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.311 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.311 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.311 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.311 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.311 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.311 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.311 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.311 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.311 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.311 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.311 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.312 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=read_operations 2024-03-13T07:35:52.312 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.312 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.312 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.312 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.312 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41890 2024-03-13T07:35:52.312 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.312 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.312 INFO:tasks.workunit.client.0.smithi132.stdout:test read_operations on pid 41890 2024-03-13T07:35:52.320 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test read_operations on pid 41890' 2024-03-13T07:35:52.320 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.321 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41890 2024-03-13T07:35:52.321 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-13T07:35:52.321 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.321 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.321 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.321 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.321 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.321 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.321 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.321 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.321 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.321 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.321 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.322 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.322 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.322 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.322 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.322 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.322 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.322 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.322 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.322 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.322 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.322 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.322 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.322 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.323 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.323 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.323 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-03-13T07:35:52.323 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s snapshots 2024-03-13T07:35:52.323 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.323 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.323 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.323 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.323 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.323 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.323 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.323 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.323 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' snapshots' 2024-03-13T07:35:52.324 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.324 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.324 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.324 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.324 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.324 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.324 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.324 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41770/exe 2024-03-13T07:35:52.324 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.324 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.324 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.324 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.324 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.324 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.325 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.325 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo snapshots 2024-03-13T07:35:52.325 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.325 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.325 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.325 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.325 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.325 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.325 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.325 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.325 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.325 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.325 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.325 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.326 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.326 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.326 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.326 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.326 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.326 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.326 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.326 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.326 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.326 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.326 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.326 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.326 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.326 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.327 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_neorados_list.log 2024-03-13T07:35:52.327 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ list: /' 2024-03-13T07:35:52.327 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.327 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.327 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.327 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.327 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.327 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.327 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.327 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.327 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.327 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.327 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.327 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.328 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.328 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.328 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.328 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_neorados_list 2024-03-13T07:35:52.328 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41790/exe 2024-03-13T07:35:52.328 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.328 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.328 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.328 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=snapshots 2024-03-13T07:35:52.328 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.328 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.328 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.328 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41916 2024-03-13T07:35:52.328 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test snapshots on pid 41916' 2024-03-13T07:35:52.329 INFO:tasks.workunit.client.0.smithi132.stdout:test snapshots on pid 41916 2024-03-13T07:35:52.338 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41916 2024-03-13T07:35:52.338 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-13T07:35:52.338 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.338 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.338 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.338 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-03-13T07:35:52.338 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.338 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.339 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.339 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.339 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.339 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.339 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.339 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.339 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s watch_notify 2024-03-13T07:35:52.339 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.339 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.339 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.339 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.339 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.339 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.339 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.339 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.339 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' watch_notify' 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.340 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.341 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.341 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.341 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.341 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.341 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.341 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.341 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.341 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.341 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.341 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_neorados_ec_list 2024-03-13T07:35:52.341 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.341 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.341 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.341 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.342 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.342 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.342 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ ec_list: /' 2024-03-13T07:35:52.342 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.342 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.342 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.342 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.342 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.342 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.342 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.342 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.342 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.342 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.342 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.342 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.343 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.343 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.343 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.343 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.343 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.343 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.343 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.343 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo watch_notify 2024-03-13T07:35:52.343 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=watch_notify 2024-03-13T07:35:52.343 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41931 2024-03-13T07:35:52.343 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test watch_notify on pid 41931' 2024-03-13T07:35:52.343 INFO:tasks.workunit.client.0.smithi132.stdout:test watch_notify on pid 41931 2024-03-13T07:35:52.353 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.353 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41931 2024-03-13T07:35:52.353 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.353 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.353 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-03-13T07:35:52.354 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.354 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.354 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.354 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.354 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.354 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.354 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.354 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.354 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-13T07:35:52.354 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.354 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.354 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.354 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.354 INFO:tasks.workunit.client.0.smithi132.stderr:++ printf %25s write_operations 2024-03-13T07:35:52.354 INFO:tasks.workunit.client.0.smithi132.stderr:+ r=' write_operations' 2024-03-13T07:35:52.355 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.355 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.355 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_neorados_ec_list.log 2024-03-13T07:35:52.355 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.355 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.355 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.355 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.355 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.355 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.355 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.355 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.355 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.355 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.355 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.355 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.356 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.356 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.356 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.356 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.356 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.356 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.356 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.356 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo write_operations 2024-03-13T07:35:52.356 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.356 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.356 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.356 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.356 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.356 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.356 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.357 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.357 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.357 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.357 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.357 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.357 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.357 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.357 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.357 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.357 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.357 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.357 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.357 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.357 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.357 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.357 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.358 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.358 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.358 INFO:tasks.workunit.client.0.smithi132.stderr:++ awk '{print $1}' 2024-03-13T07:35:52.358 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.358 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.358 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.358 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.358 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.358 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.358 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.358 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.358 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.358 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.358 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.358 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.359 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.359 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.359 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.359 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.359 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.359 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.359 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.359 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.359 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.359 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.359 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.359 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.359 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.359 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.359 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.359 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.360 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.360 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.360 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.360 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.360 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.360 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.360 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.360 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.360 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.360 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.360 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.360 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.360 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.360 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.360 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.361 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.361 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.361 INFO:tasks.workunit.client.0.smithi132.stderr:+ ff=write_operations 2024-03-13T07:35:52.361 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.361 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41947 2024-03-13T07:35:52.361 INFO:tasks.workunit.client.0.smithi132.stdout:test write_operations on pid 41947 2024-03-13T07:35:52.396 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'test write_operations on pid 41947' 2024-03-13T07:35:52.396 INFO:tasks.workunit.client.0.smithi132.stderr:+ pids[$f]=41947 2024-03-13T07:35:52.396 INFO:tasks.workunit.client.0.smithi132.stderr:+ ret=0 2024-03-13T07:35:52.397 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' 1 -eq 1 ']' 2024-03-13T07:35:52.397 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.397 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T07:35:52.397 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40922 2024-03-13T07:35:52.397 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 40922 2024-03-13T07:35:52.397 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.397 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.397 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.397 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.397 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.397 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.397 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.397 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.397 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.397 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.398 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.398 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.398 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.398 INFO:tasks.workunit.client.0.smithi132.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-03-13T07:35:52.398 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.398 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.398 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.398 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.398 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.398 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.398 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.398 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.398 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -f /etc/bashrc ']' 2024-03-13T07:35:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:+ . /etc/bashrc 2024-03-13T07:35:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -z '' ']' 2024-03-13T07:35:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:++ BASHRCSOURCED=Y 2024-03-13T07:35:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:++ shopt -q login_shell 2024-03-13T07:35:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.402 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.402 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.402 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.402 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.402 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.402 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.402 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.402 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.402 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.402 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.402 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.402 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.402 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.402 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.403 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.403 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.403 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.403 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.403 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.403 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.403 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.403 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.403 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.403 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.403 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.403 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.403 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.403 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.403 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.403 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.404 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.404 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.404 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.404 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.404 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.404 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.404 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.404 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.404 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.404 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.404 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.404 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.404 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.404 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.404 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.404 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.405 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.405 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.405 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.405 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.405 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.405 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.405 INFO:tasks.workunit.client.0.smithi132.stderr:+++ umask 2024-03-13T07:35:52.405 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.405 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41868/exe 2024-03-13T07:35:52.405 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' 0022 -eq 0 ']' 2024-03-13T07:35:52.405 INFO:tasks.workunit.client.0.smithi132.stderr:++ SHELL=/bin/bash 2024-03-13T07:35:52.405 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.405 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.406 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-13T07:35:52.406 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.406 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-13T07:35:52.406 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-13T07:35:52.406 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.406 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-13T07:35:52.406 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.406 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-13T07:35:52.406 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.406 INFO:tasks.workunit.client.0.smithi132.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-13T07:35:52.406 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-13T07:35:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-13T07:35:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41832/exe 2024-03-13T07:35:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_neorados_pool 2024-03-13T07:35:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_neorados_misc.log 2024-03-13T07:35:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ misc: /' 2024-03-13T07:35:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_neorados_misc 2024-03-13T07:35:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_neorados_pool.log 2024-03-13T07:35:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ pool: /' 2024-03-13T07:35:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'grep=grep --color=auto' 2024-03-13T07:35:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-13T07:35:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-13T07:35:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-13T07:35:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorls.sh 2024-03-13T07:35:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' '!' -t 0 ']' 2024-03-13T07:35:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:+++ return 2024-03-13T07:35:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-13T07:35:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-13T07:35:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export S_COLORS=auto 2024-03-13T07:35:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:+++ S_COLORS=auto 2024-03-13T07:35:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-13T07:35:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-13T07:35:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41890/exe 2024-03-13T07:35:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41916/exe 2024-03-13T07:35:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_neorados_read_operations 2024-03-13T07:35:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-13T07:35:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_neorados_read_operations.log 2024-03-13T07:35:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-13T07:35:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-13T07:35:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-13T07:35:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ read_operations: /' 2024-03-13T07:35:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-13T07:35:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-13T07:35:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-13T07:35:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41931/exe 2024-03-13T07:35:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ watch_notify: /' 2024-03-13T07:35:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ snapshots: /' 2024-03-13T07:35:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_neorados_snapshots.log 2024-03-13T07:35:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_neorados_snapshots 2024-03-13T07:35:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_neorados_watch_notify 2024-03-13T07:35:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-03-13T07:35:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-13T07:35:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-13T07:35:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-13T07:35:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-13T07:35:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-13T07:35:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-13T07:35:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-13T07:35:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/gawk.sh 2024-03-13T07:35:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-13T07:35:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/lang.sh 2024-03-13T07:35:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup 2024-03-13T07:35:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/locale 2024-03-13T07:35:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-13T07:35:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-13T07:35:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-13T07:35:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-13T07:35:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-13T07:35:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:++++ export LANG=en_US.UTF-8 2024-03-13T07:35:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:++++ LANG=en_US.UTF-8 2024-03-13T07:35:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-13T07:35:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-13T07:35:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:+++ LANG=en_US.UTF-8 2024-03-13T07:35:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:+++ unset LANG_backup config 2024-03-13T07:35:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n '' ']' 2024-03-13T07:35:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-13T07:35:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' dumb = linux ']' 2024-03-13T07:35:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-13T07:35:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/less.sh 2024-03-13T07:35:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-13T07:35:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:++ for i in /etc/profile.d/*.sh 2024-03-13T07:35:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-13T07:35:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:++ '[' '' ']' 2024-03-13T07:35:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:++ . /etc/profile.d/which2.sh 2024-03-13T07:35:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-13T07:35:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:+++++ readlink /proc/41947/exe 2024-03-13T07:35:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:++++ basename /usr/bin/bash 2024-03-13T07:35:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_declare='declare -f' 2024-03-13T07:35:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:+++ which_opt=-f 2024-03-13T07:35:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export which_declare 2024-03-13T07:35:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:+++ export -f which 2024-03-13T07:35:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset i 2024-03-13T07:35:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:++ unset -f pathmunge 2024-03-13T07:35:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-13T07:35:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:+ export PATH 2024-03-13T07:35:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-13T07:35:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:+ unset rc 2024-03-13T07:35:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph_test_neorados_write_operations 2024-03-13T07:35:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:+ tee ceph_test_neorados_write_operations.log 2024-03-13T07:35:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:+ sed 's/^/ write_operations: /' 2024-03-13T07:35:53.378 INFO:tasks.workunit.client.0.smithi132.stdout:0/2636667481 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f71c006b7d0 0x7f71c006dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:35:53.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.913+0000 7f71de6f5640 10 monclient: get_auth_request con 0x7f71c006b7d0 auth_method 0 2024-03-13T07:35:53.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.914+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f71d8055960 con 0x55a2c179ada0 2024-03-13T07:35:53.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.914+0000 7f71cf7fe640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_map(26..26 src has 1..26) v4 2024-03-13T07:35:53.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.914+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map got epochs [26,26] > 0 2024-03-13T07:35:53.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.914+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding full epoch 26 2024-03-13T07:35:53.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.914+0000 7f71cf7fe640 20 client.4373.objecter dump_active .. 0 homeless 2024-03-13T07:35:53.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.914+0000 7f71e0bf5780 10 librados: wait_for_osdmap done waiting 2024-03-13T07:35:53.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.914+0000 7f71e0bf5780 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:35:53.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.914+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f71d4008c00 con 0x55a2c179ada0 2024-03-13T07:35:53.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.915+0000 7f71de6f5640 1 --2- 172.21.15.132:0/2636667481 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f71c006b7d0 0x7f71c006dc90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f71d0054000 tx=0x7f71d004c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:35:53.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.915+0000 7f71cf7fe640 10 client.4373.objecter ms_handle_connect 0x7f71c006b7d0 2024-03-13T07:35:53.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.969+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71d8053b60 con 0x55a2c179ada0 2024-03-13T07:35:53.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.969+0000 7f71cf7fe640 10 monclient: handle_get_version_reply finishing 1 version 26 2024-03-13T07:35:53.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.969+0000 7f71cffff640 10 client.4373.objecter _get_latest_version latest 26, have it 2024-03-13T07:35:53.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.969+0000 7f71e0bf5780 10 client.4373.objecter create_pool name=ceph_test_rados_api_asio 2024-03-13T07:35:53.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.969+0000 7f71e0bf5780 10 client.4373.objecter pool_op_submit 1 2024-03-13T07:35:53.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.969+0000 7f71e0bf5780 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:35:53.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:51.969+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- pool_op(create pool 0 tid 1 name ceph_test_rados_api_asio v0) v4 -- 0x55a2c167b4f0 con 0x55a2c179ada0 2024-03-13T07:35:53.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.378+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 <== mon.2 v2:172.21.15.132:3301/0 7 ==== pool_op_reply(tid 1 (0) Success v27) v1 ==== 43+0+0 (secure 0 0 0) 0x7f71d80578c0 con 0x55a2c179ada0 2024-03-13T07:35:53.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.378+0000 7f71cf7fe640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 pool_op_reply(tid 1 (0) Success v27) v1 2024-03-13T07:35:53.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.378+0000 7f71cf7fe640 10 client.4373.objecter handle_pool_op_reply pool_op_reply(tid 1 (0) Success v27) v1 2024-03-13T07:35:53.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.378+0000 7f71cf7fe640 10 client.4373.objecter have request 1 at 0x55a2c183ad40 Op: create 2024-03-13T07:35:53.445 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.378+0000 7f71cf7fe640 20 client.4373.objecter waiting for client to reach epoch 27 before calling back 2024-03-13T07:35:53.445 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.378+0000 7f71cf7fe640 10 client.4373.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T07:35:53.445 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.378+0000 7f71cf7fe640 10 monclient: _renew_subs 2024-03-13T07:35:53.445 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.378+0000 7f71cf7fe640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T07:35:53.445 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.378+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=27}) v3 -- 0x7f71c0071540 con 0x55a2c179ada0 2024-03-13T07:35:53.445 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.378+0000 7f71cf7fe640 10 client.4373.objecter done 2024-03-13T07:35:53.445 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.387+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 <== mon.2 v2:172.21.15.132:3301/0 8 ==== osd_map(27..27 src has 1..27) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f71d8055bc0 con 0x55a2c179ada0 2024-03-13T07:35:53.446 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.387+0000 7f71cf7fe640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_map(27..27 src has 1..27) v4 2024-03-13T07:35:53.446 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.387+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map got epochs [27,27] > 26 2024-03-13T07:35:53.446 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.387+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 27 2024-03-13T07:35:53.446 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.387+0000 7f71cf7fe640 20 client.4373.objecter dump_active .. 0 homeless 2024-03-13T07:35:53.446 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.387+0000 7f71e0bf5780 10 librados: set snap read head -> 1 2024-03-13T07:35:53.446 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.388+0000 7f71e0bf5780 10 librados: writefull oid=exist nspace= 2024-03-13T07:35:53.446 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.388+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c16c6890 2024-03-13T07:35:53.446 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.388+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 27 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:35:53.446 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.388+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:35:53.446 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.388+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.d3a59201 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:35:53.446 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.388+0000 7f71e0bf5780 1 --2- 172.21.15.132:0/2636667481 >> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] conn(0x55a2c16c5ed0 0x55a2c1846ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T07:35:53.446 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.388+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:35:53.446 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.388+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid exist '@9' '@9' [writefull 0~5 in=5b] tid 2 osd.7 2024-03-13T07:35:53.447 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.388+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:35:53.447 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.388+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 2 2024-03-13T07:35:53.447 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.388+0000 7f71e0bf5780 15 client.4373.objecter _send_op 2 to 9.1 on osd.7 2024-03-13T07:35:53.447 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.388+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:2 9.1 9:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) v9 -- 0x55a2c16c62b0 con 0x55a2c16c5ed0 2024-03-13T07:35:53.447 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.388+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:35:53.447 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.388+0000 7f71e0bf5780 5 client.4373.objecter 1 in flight 2024-03-13T07:36:00.447 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.445+0000 7f71ddef4640 1 --2- 172.21.15.132:0/2636667481 >> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] conn(0x55a2c16c5ed0 0x55a2c1846ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T07:36:00.448 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.610+0000 7f71ddef4640 10 monclient: get_auth_request con 0x55a2c16c5ed0 auth_method 0 2024-03-13T07:36:00.448 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.770+0000 7f71ddef4640 1 --2- 172.21.15.132:0/2636667481 >> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] conn(0x55a2c16c5ed0 0x55a2c1846ef0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T07:36:00.448 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:53.770+0000 7f71cf7fe640 10 client.4373.objecter ms_handle_connect 0x55a2c16c5ed0 2024-03-13T07:36:00.448 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:54.792+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:00.448 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:54.792+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:00.448 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:54.792+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:00.448 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:54.792+0000 7f71ce7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:36:00.448 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:55.792+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:00.448 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:55.792+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:00.448 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:55.792+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:00.448 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:56.793+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:00.448 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:56.793+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:00.449 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:56.793+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:00.449 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:56.846+0000 7f71e0959640 10 client.4373.objecter tick 2024-03-13T07:36:00.449 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:57.793+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:00.449 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:57.793+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:00.449 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:57.793+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:00.449 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:58.793+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:00.449 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:58.793+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:00.449 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:58.793+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:00.449 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:59.794+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:00.449 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:59.794+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:00.449 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:35:59.794+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:00.449 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:00.447+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 1 ==== osd_map(28..28 src has 1..28) v4 ==== 13694+0+0 (crc 0 0 0) 0x7f71d4008840 con 0x55a2c16c5ed0 2024-03-13T07:36:00.449 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:00.447+0000 7f71cf7fe640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_map(28..28 src has 1..28) v4 2024-03-13T07:36:00.449 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:00.447+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map got epochs [28,28] > 27 2024-03-13T07:36:00.449 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:00.447+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 28 2024-03-13T07:36:00.450 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:00.448+0000 7f71cf7fe640 10 client.4373.objecter checking op 2 2024-03-13T07:36:00.450 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:00.448+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 28 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:00.450 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:00.448+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:02.596 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:00.448+0000 7f71cf7fe640 20 client.4373.objecter dump_active .. 0 homeless 2024-03-13T07:36:02.596 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:00.448+0000 7f71cf7fe640 20 client.4373.objecter 2 9.d3a59201 osd.7 exist [writefull 0~5 in=5b] 2024-03-13T07:36:02.596 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:00.794+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:02.596 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:00.794+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:02.596 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:00.794+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:02.596 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:01.794+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:02.596 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:01.794+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:02.596 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:01.794+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:02.596 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:01.847+0000 7f71e0959640 10 client.4373.objecter tick 2024-03-13T07:36:02.596 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.595+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 2 ==== osd_op_reply(2 exist [writefull 0~5] v28'1 uv1 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:02.596 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.595+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(2 exist [writefull 0~5] v28'1 uv1 ondisk = 0) v8 2024-03-13T07:36:02.596 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.595+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:02.597 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.595+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 2 ondisk uv 1 in 9.1 attempt 0 2024-03-13T07:36:02.597 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.595+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 0 2024-03-13T07:36:02.597 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.595+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 2 2024-03-13T07:36:02.597 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.595+0000 7f71ddef4640 15 client.4373.objecter _finish_op 2 2024-03-13T07:36:02.597 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.595+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:02.597 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.595+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 2 2024-03-13T07:36:02.597 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.595+0000 7f71ddef4640 5 client.4373.objecter 0 in flight 2024-03-13T07:36:02.597 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-03-13T07:36:02.597 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 10 librados: Objecter returned from writefull r=0 2024-03-13T07:36:02.597 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c1839520 2024-03-13T07:36:02.597 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 28 base exist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:02.597 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:02.597 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.d3a59201 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:02.597 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:02.597 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid exist '@9' '@9' [read 0~256] tid 3 osd.7 2024-03-13T07:36:02.598 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:02.598 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 3 2024-03-13T07:36:02.598 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 15 client.4373.objecter _send_op 3 to 9.1 on osd.7 2024-03-13T07:36:02.598 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:3 9.1 9:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v9 -- 0x55a2c16c69b0 con 0x55a2c16c5ed0 2024-03-13T07:36:03.215 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:03.216 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 5 client.4373.objecter 1 in flight 2024-03-13T07:36:03.216 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c18399d0 2024-03-13T07:36:03.216 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 28 base noexist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:03.216 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target noexist @9 -> pgid 9.a7bbaa1 2024-03-13T07:36:03.216 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.a7bbaa1 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:03.216 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:03.216 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid noexist '@9' '@9' [read 0~256] tid 4 osd.7 2024-03-13T07:36:03.216 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:03.216 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 4 2024-03-13T07:36:03.216 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 15 client.4373.objecter _send_op 4 to 9.1 on osd.7 2024-03-13T07:36:03.216 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:4 9.1 9:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v9 -- 0x55a2c1839e80 con 0x55a2c16c5ed0 2024-03-13T07:36:03.216 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 5 2024-03-13T07:36:03.216 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.596+0000 7f71e0bf5780 5 client.4373.objecter 2 in flight 2024-03-13T07:36:03.216 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.794+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:03.217 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.794+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:03.217 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:02.794+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:03.217 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.215+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 3 ==== osd_map(29..31 src has 1..33) v4 ==== 10343+0+0 (crc 0 0 0) 0x7f71d40136d0 con 0x55a2c16c5ed0 2024-03-13T07:36:03.217 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.215+0000 7f71cf7fe640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_map(29..31 src has 1..33) v4 2024-03-13T07:36:03.217 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.215+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map got epochs [29,31] > 28 2024-03-13T07:36:03.217 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.215+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 29 2024-03-13T07:36:03.217 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.215+0000 7f71cf7fe640 10 client.4373.objecter checking op 3 2024-03-13T07:36:03.217 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.215+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 29 base exist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:03.217 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:03.217 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 10 client.4373.objecter checking op 4 2024-03-13T07:36:03.217 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 29 base noexist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:03.217 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target noexist @9 -> pgid 9.a7bbaa1 2024-03-13T07:36:03.217 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 30 2024-03-13T07:36:03.217 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 10 client.4373.objecter checking op 3 2024-03-13T07:36:03.227 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 30 base exist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:03.227 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:03.227 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 10 client.4373.objecter checking op 4 2024-03-13T07:36:03.227 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 30 base noexist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:03.227 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target noexist @9 -> pgid 9.a7bbaa1 2024-03-13T07:36:03.227 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 31 2024-03-13T07:36:03.227 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 10 client.4373.objecter checking op 3 2024-03-13T07:36:03.228 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 31 base exist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:03.228 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:03.228 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 10 client.4373.objecter checking op 4 2024-03-13T07:36:03.228 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 31 base noexist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:03.228 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target noexist @9 -> pgid 9.a7bbaa1 2024-03-13T07:36:03.228 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 20 client.4373.objecter dump_active .. 0 homeless 2024-03-13T07:36:03.228 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 20 client.4373.objecter 3 9.d3a59201 osd.7 exist [read 0~256] 2024-03-13T07:36:03.228 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.216+0000 7f71cf7fe640 20 client.4373.objecter 4 9.a7bbaa1 osd.7 noexist [read 0~256] 2024-03-13T07:36:03.228 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 4 ==== osd_op_reply(3 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:03.228 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(3 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 2024-03-13T07:36:03.228 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:03.228 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 3 ondisk uv 1 in 9.1 attempt 0 2024-03-13T07:36:03.228 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 5 2024-03-13T07:36:03.229 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 3 2024-03-13T07:36:03.229 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 15 client.4373.objecter _finish_op 3 2024-03-13T07:36:03.229 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 5 2024-03-13T07:36:03.229 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 3 2024-03-13T07:36:03.229 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 5 client.4373.objecter 1 in flight 2024-03-13T07:36:03.229 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 5 ==== osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:03.229 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2024-03-13T07:36:03.229 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:03.229 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (632 ms) 2024-03-13T07:36:03.229 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 4 ondisk uv 0 in 9.1 attempt 0 2024-03-13T07:36:03.229 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-03-13T07:36:03.229 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 10 client.4373.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-03-13T07:36:03.229 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 0 2024-03-13T07:36:03.230 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 4 2024-03-13T07:36:03.230 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 15 client.4373.objecter _finish_op 4 2024-03-13T07:36:03.230 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:03.230 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 4 2024-03-13T07:36:03.230 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71ddef4640 5 client.4373.objecter 0 in flight 2024-03-13T07:36:03.230 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c1839520 2024-03-13T07:36:03.230 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 31 base exist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:03.230 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:03.230 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.d3a59201 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:03.230 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:03.230 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid exist '@9' '@9' [read 0~256] tid 5 osd.7 2024-03-13T07:36:03.230 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:03.230 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 5 2024-03-13T07:36:03.230 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 15 client.4373.objecter _send_op 5 to 9.1 on osd.7 2024-03-13T07:36:03.231 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:5 9.1 9:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v9 -- 0x55a2c18399d0 con 0x55a2c16c5ed0 2024-03-13T07:36:03.231 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:03.231 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 5 client.4373.objecter 1 in flight 2024-03-13T07:36:03.231 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c1847430 2024-03-13T07:36:03.231 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 31 base noexist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:03.231 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target noexist @9 -> pgid 9.a7bbaa1 2024-03-13T07:36:03.231 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.a7bbaa1 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:03.231 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:03.231 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid noexist '@9' '@9' [read 0~256] tid 6 osd.7 2024-03-13T07:36:03.231 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:03.816 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 6 2024-03-13T07:36:03.816 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 15 client.4373.objecter _send_op 6 to 9.1 on osd.7 2024-03-13T07:36:03.816 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:6 9.1 9:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v9 -- 0x55a2c18478e0 con 0x55a2c16c5ed0 2024-03-13T07:36:03.816 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 5 2024-03-13T07:36:03.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.227+0000 7f71e0bf5780 5 client.4373.objecter 2 in flight 2024-03-13T07:36:03.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.794+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:03.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.794+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:03.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.794+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:03.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (589 ms) 2024-03-13T07:36:03.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-03-13T07:36:03.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 6 ==== osd_op_reply(5 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:03.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(5 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 2024-03-13T07:36:03.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:03.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 5 ondisk uv 1 in 9.1 attempt 0 2024-03-13T07:36:03.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 5 2024-03-13T07:36:03.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 5 2024-03-13T07:36:03.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 15 client.4373.objecter _finish_op 5 2024-03-13T07:36:03.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 5 2024-03-13T07:36:03.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 5 2024-03-13T07:36:03.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 5 client.4373.objecter 1 in flight 2024-03-13T07:36:03.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 7 ==== osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:03.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2024-03-13T07:36:03.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:03.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 6 ondisk uv 0 in 9.1 attempt 0 2024-03-13T07:36:03.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 10 client.4373.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-03-13T07:36:03.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 0 2024-03-13T07:36:03.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 6 2024-03-13T07:36:03.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 15 client.4373.objecter _finish_op 6 2024-03-13T07:36:03.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:04.432 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 6 2024-03-13T07:36:04.432 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71ddef4640 5 client.4373.objecter 0 in flight 2024-03-13T07:36:04.432 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c1847430 2024-03-13T07:36:04.432 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 31 base exist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:04.432 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:04.432 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.d3a59201 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:04.432 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:04.432 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid exist '@9' '@9' [read 0~256] tid 7 osd.7 2024-03-13T07:36:04.432 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:04.432 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 7 2024-03-13T07:36:04.432 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 15 client.4373.objecter _send_op 7 to 9.1 on osd.7 2024-03-13T07:36:04.432 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:7 9.1 9:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v9 -- 0x55a2c1839520 con 0x55a2c16c5ed0 2024-03-13T07:36:04.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:04.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 5 client.4373.objecter 1 in flight 2024-03-13T07:36:04.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c1888310 2024-03-13T07:36:04.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 31 base noexist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:04.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target noexist @9 -> pgid 9.a7bbaa1 2024-03-13T07:36:04.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.a7bbaa1 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:04.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:04.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid noexist '@9' '@9' [read 0~256] tid 8 osd.7 2024-03-13T07:36:04.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:04.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 8 2024-03-13T07:36:04.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 15 client.4373.objecter _send_op 8 to 9.1 on osd.7 2024-03-13T07:36:04.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:8 9.1 9:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v9 -- 0x55a2c18887c0 con 0x55a2c16c5ed0 2024-03-13T07:36:04.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 5 2024-03-13T07:36:04.434 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:03.816+0000 7f71e0bf5780 5 client.4373.objecter 2 in flight 2024-03-13T07:36:04.434 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 8 ==== osd_map(32..33 src has 1..33) v4 ==== 2406+0+0 (crc 0 0 0) 0x7f71d40138d0 con 0x55a2c16c5ed0 2024-03-13T07:36:04.493 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_map(32..33 src has 1..33) v4 2024-03-13T07:36:04.493 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map got epochs [32,33] > 31 2024-03-13T07:36:04.493 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 32 2024-03-13T07:36:04.493 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 10 client.4373.objecter checking op 7 2024-03-13T07:36:04.493 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 32 base exist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:04.493 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:04.493 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 10 client.4373.objecter checking op 8 2024-03-13T07:36:04.493 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 32 base noexist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:04.493 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target noexist @9 -> pgid 9.a7bbaa1 2024-03-13T07:36:04.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 33 2024-03-13T07:36:04.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 10 client.4373.objecter checking op 7 2024-03-13T07:36:04.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 33 base exist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:04.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:04.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 10 client.4373.objecter checking op 8 2024-03-13T07:36:04.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 33 base noexist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:04.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target noexist @9 -> pgid 9.a7bbaa1 2024-03-13T07:36:04.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 20 client.4373.objecter dump_active .. 0 homeless 2024-03-13T07:36:04.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 20 client.4373.objecter 7 9.d3a59201 osd.7 exist [read 0~256] 2024-03-13T07:36:04.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.431+0000 7f71cf7fe640 20 client.4373.objecter 8 9.a7bbaa1 osd.7 noexist [read 0~256] 2024-03-13T07:36:04.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.468+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 9 ==== osd_op_reply(7 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:04.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.468+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(7 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 2024-03-13T07:36:04.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.468+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:04.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.468+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 7 ondisk uv 1 in 9.1 attempt 0 2024-03-13T07:36:04.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.468+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 5 2024-03-13T07:36:04.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.468+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 7 2024-03-13T07:36:04.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.468+0000 7f71ddef4640 15 client.4373.objecter _finish_op 7 2024-03-13T07:36:04.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.468+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 5 2024-03-13T07:36:04.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.468+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 7 2024-03-13T07:36:04.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.468+0000 7f71ddef4640 5 client.4373.objecter 1 in flight 2024-03-13T07:36:04.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 10 ==== osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:04.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2024-03-13T07:36:04.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:04.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 8 ondisk uv 0 in 9.1 attempt 0 2024-03-13T07:36:04.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71ddef4640 10 client.4373.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-03-13T07:36:04.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 0 2024-03-13T07:36:04.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 8 2024-03-13T07:36:04.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71ddef4640 15 client.4373.objecter _finish_op 8 2024-03-13T07:36:04.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:04.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 8 2024-03-13T07:36:04.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71ddef4640 5 client.4373.objecter 0 in flight 2024-03-13T07:36:04.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (677 ms) 2024-03-13T07:36:04.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-03-13T07:36:04.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71e0bf5780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2024-03-13T07:36:04.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71e0bf5780 20 librados: queue_aio_write 0x55a2c1637a70 completion 0x55a2c16137b0 write_seq 1 2024-03-13T07:36:04.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c1847430 2024-03-13T07:36:04.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 33 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:04.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:04.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.d3a59201 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:04.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:04.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid exist '@9' '@9' [write 0~5 in=5b] tid 9 osd.7 2024-03-13T07:36:04.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:04.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 9 2024-03-13T07:36:04.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71e0bf5780 15 client.4373.objecter _send_op 9 to 9.1 on osd.7 2024-03-13T07:36:04.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:9 9.1 9:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v9 -- 0x55a2c1848c90 con 0x55a2c16c5ed0 2024-03-13T07:36:04.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:04.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71e0bf5780 5 client.4373.objecter 1 in flight 2024-03-13T07:36:04.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.492+0000 7f71e0bf5780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2024-03-13T07:36:04.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.793+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:04.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.793+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:06.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.793+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:06.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:04.793+0000 7f71ce7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:36:06.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:05.793+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:06.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:05.793+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:06.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:05.793+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:06.817 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.793+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:06.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.793+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:06.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.793+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:06.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 11 ==== osd_op_reply(9 exist [write 0~5] v33'2 uv2 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:06.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(9 exist [write 0~5] v33'2 uv2 ondisk = 0) v8 2024-03-13T07:36:06.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:06.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 9 ondisk uv 2 in 9.1 attempt 0 2024-03-13T07:36:06.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 0 2024-03-13T07:36:06.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 9 2024-03-13T07:36:06.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71ddef4640 15 client.4373.objecter _finish_op 9 2024-03-13T07:36:06.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:06.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 9 2024-03-13T07:36:06.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71ddef4640 5 client.4373.objecter 0 in flight 2024-03-13T07:36:06.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71ddef4640 20 librados: complete_aio_write 0x55a2c16137b0 2024-03-13T07:36:06.818 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (2324 ms) 2024-03-13T07:36:06.819 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-03-13T07:36:06.819 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71e0bf5780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2024-03-13T07:36:06.819 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71e0bf5780 20 librados: queue_aio_write 0x55a2c1637a70 completion 0x55a2c1602ae0 write_seq 2 2024-03-13T07:36:06.819 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c1847430 2024-03-13T07:36:06.819 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 33 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:06.819 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:06.819 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.d3a59201 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:06.819 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:06.819 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid exist '@9' '@9' [write 0~5 in=5b] tid 10 osd.7 2024-03-13T07:36:06.819 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:06.819 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 10 2024-03-13T07:36:06.819 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71e0bf5780 15 client.4373.objecter _send_op 10 to 9.1 on osd.7 2024-03-13T07:36:11.416 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:02:35.306 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:11.421 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:02:35.311 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:11.421 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:02:35.312 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:11.485 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:02:35.376 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:11.503 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:02:35.373 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:11.513 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:02:35.384 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:11.721 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:02:35.611 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:11.757 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:02:35.647 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:11.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:10 9.1 9:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v9 -- 0x55a2c1848c90 con 0x55a2c16c5ed0 2024-03-13T07:36:11.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:11.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71e0bf5780 5 client.4373.objecter 1 in flight 2024-03-13T07:36:11.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.816+0000 7f71e0bf5780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2024-03-13T07:36:11.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.847+0000 7f71e0959640 10 client.4373.objecter tick 2024-03-13T07:36:11.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.849+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 12 ==== osd_map(34..35 src has 1..35) v4 ==== 4201+0+0 (crc 0 0 0) 0x7f71d4013ad0 con 0x55a2c16c5ed0 2024-03-13T07:36:11.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.849+0000 7f71cf7fe640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_map(34..35 src has 1..35) v4 2024-03-13T07:36:11.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.849+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map got epochs [34,35] > 33 2024-03-13T07:36:11.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.849+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 34 2024-03-13T07:36:11.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.849+0000 7f71cf7fe640 10 client.4373.objecter checking op 10 2024-03-13T07:36:11.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.849+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 34 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:11.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.849+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:11.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.849+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 35 2024-03-13T07:36:11.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.849+0000 7f71cf7fe640 10 client.4373.objecter checking op 10 2024-03-13T07:36:11.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.849+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 35 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:11.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.849+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:11.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.849+0000 7f71cf7fe640 20 client.4373.objecter dump_active .. 0 homeless 2024-03-13T07:36:11.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:06.849+0000 7f71cf7fe640 20 client.4373.objecter 10 9.d3a59201 osd.7 exist [write 0~5 in=5b] 2024-03-13T07:36:11.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:07.793+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:11.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:07.793+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:11.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:07.793+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:11.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:08.793+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:11.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:08.793+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:11.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:08.793+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:11.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:09.793+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:11.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:09.793+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:11.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:09.793+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:11.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:10.793+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:11.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:10.794+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:11.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:10.794+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:11.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:11.794+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:11.840 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:02:35.711 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:14.839 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:11.794+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:14.839 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:11.794+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:14.839 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:11.848+0000 7f71e0959640 10 client.4373.objecter tick 2024-03-13T07:36:14.839 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:12.794+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:14.839 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:12.794+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:14.839 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:12.794+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:14.839 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:13.794+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:14.839 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:13.794+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:14.840 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:13.794+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:14.840 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.794+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:14.840 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.794+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:14.840 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.794+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:14.840 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.794+0000 7f71ce7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T07:36:14.840 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 13 ==== osd_op_reply(10 exist [write 0~5] v35'3 uv3 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:14.840 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(10 exist [write 0~5] v35'3 uv3 ondisk = 0) v8 2024-03-13T07:36:14.840 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:14.840 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 10 ondisk uv 3 in 9.1 attempt 0 2024-03-13T07:36:14.840 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 0 2024-03-13T07:36:14.840 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 10 2024-03-13T07:36:14.840 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71ddef4640 15 client.4373.objecter _finish_op 10 2024-03-13T07:36:14.840 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:14.840 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 10 2024-03-13T07:36:14.840 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71ddef4640 5 client.4373.objecter 0 in flight 2024-03-13T07:36:14.841 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71ddef4640 20 librados: complete_aio_write 0x55a2c1602ae0 2024-03-13T07:36:14.841 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (8022 ms) 2024-03-13T07:36:14.841 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-03-13T07:36:14.841 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71e0bf5780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2024-03-13T07:36:14.841 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71e0bf5780 20 librados: queue_aio_write 0x55a2c1637a70 completion 0x55a2c1847430 write_seq 3 2024-03-13T07:36:14.841 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c186b880 2024-03-13T07:36:14.841 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 35 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:14.841 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:14.841 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.d3a59201 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:14.841 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:14.939 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid exist '@9' '@9' [write 0~5 in=5b] tid 11 osd.7 2024-03-13T07:36:14.940 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:14.940 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 11 2024-03-13T07:36:14.940 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71e0bf5780 15 client.4373.objecter _send_op 11 to 9.1 on osd.7 2024-03-13T07:36:14.940 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:11 9.1 9:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v9 -- 0x55a2c186bd30 con 0x55a2c16c5ed0 2024-03-13T07:36:14.940 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:14.940 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71e0bf5780 5 client.4373.objecter 1 in flight 2024-03-13T07:36:14.940 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.838+0000 7f71e0bf5780 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2024-03-13T07:36:14.940 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 14 ==== osd_map(36..41 src has 1..41) v4 ==== 5461+0+0 (crc 0 0 0) 0x7f71d4013ad0 con 0x55a2c16c5ed0 2024-03-13T07:36:14.940 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_map(36..41 src has 1..41) v4 2024-03-13T07:36:14.940 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map got epochs [36,41] > 35 2024-03-13T07:36:14.940 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 36 2024-03-13T07:36:14.940 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 10 client.4373.objecter checking op 11 2024-03-13T07:36:14.941 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 36 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:14.941 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:14.941 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 37 2024-03-13T07:36:14.941 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 10 client.4373.objecter checking op 11 2024-03-13T07:36:14.941 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 37 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:14.941 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:14.941 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 38 2024-03-13T07:36:14.941 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 10 client.4373.objecter checking op 11 2024-03-13T07:36:14.941 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 38 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:14.941 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:14.941 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 39 2024-03-13T07:36:14.941 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 10 client.4373.objecter checking op 11 2024-03-13T07:36:14.941 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 39 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:14.941 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:14.941 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 40 2024-03-13T07:36:16.184 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 10 client.4373.ob handler_error: Running main() from gmock_main.cc 2024-03-13T07:36:16.184 INFO:tasks.workunit.client.0.smithi132.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-03-13T07:36:16.184 INFO:tasks.workunit.client.0.smithi132.stdout: handler_error: [----------] Global test environment set-up. 2024-03-13T07:36:16.184 INFO:tasks.workunit.client.0.smithi132.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-03-13T07:36:16.184 INFO:tasks.workunit.client.0.smithi132.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-03-13T07:36:16.184 INFO:tasks.workunit.client.0.smithi132.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (24119 ms) 2024-03-13T07:36:16.184 INFO:tasks.workunit.client.0.smithi132.stdout: handler_error: [----------] 1 test from neocls_handler_error (24119 ms total) 2024-03-13T07:36:16.184 INFO:tasks.workunit.client.0.smithi132.stdout: handler_error: 2024-03-13T07:36:16.184 INFO:tasks.workunit.client.0.smithi132.stdout: handler_error: [----------] Global test environment tear-down 2024-03-13T07:36:16.184 INFO:tasks.workunit.client.0.smithi132.stdout: handler_error: [==========] 1 test from 1 test suite ran. (24119 ms total) 2024-03-13T07:36:16.184 INFO:tasks.workunit.client.0.smithi132.stdout: handler_error: [ PASSED ] 1 test. 2024-03-13T07:36:17.002 INFO:tasks.workunit.client.0.smithi132.stdout:jecter checking op 11 2024-03-13T07:36:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 40 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 41 2024-03-13T07:36:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 10 client.4373.objecter checking op 11 2024-03-13T07:36:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 41 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 20 client.4373.objecter dump_active .. 0 homeless 2024-03-13T07:36:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:14.939+0000 7f71cf7fe640 20 client.4373.objecter 11 9.d3a59201 osd.7 exist [write 0~5 in=5b] 2024-03-13T07:36:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:15.795+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:15.795+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:17.003 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:15.795+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:17.003 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:16.795+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:17.003 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:16.795+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:17.003 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:16.795+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:17.003 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:16.849+0000 7f71e0959640 10 client.4373.objecter tick 2024-03-13T07:36:17.003 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 15 ==== osd_op_reply(11 exist [write 0~5] v41'4 uv4 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:17.003 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(11 exist [write 0~5] v41'4 uv4 ondisk = 0) v8 2024-03-13T07:36:17.003 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:17.003 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 11 ondisk uv 4 in 9.1 attempt 0 2024-03-13T07:36:17.003 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 0 2024-03-13T07:36:17.003 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 11 2024-03-13T07:36:17.003 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71ddef4640 15 client.4373.objecter _finish_op 11 2024-03-13T07:36:17.003 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:17.003 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 11 2024-03-13T07:36:17.004 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71ddef4640 5 client.4373.objecter 0 in flight 2024-03-13T07:36:17.004 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71ddef4640 20 librados: complete_aio_write 0x55a2c1847430 2024-03-13T07:36:17.004 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (2163 ms) 2024-03-13T07:36:17.004 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-03-13T07:36:17.004 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c1847c80 2024-03-13T07:36:17.004 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 41 base exist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:17.004 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:17.482 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.d3a59201 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:17.483 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:17.483 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid exist '@9' '@9' [read 0~0] tid 12 osd.7 2024-03-13T07:36:17.483 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:17.483 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 12 2024-03-13T07:36:17.483 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 15 client.4373.objecter _send_op 12 to 9.1 on osd.7 2024-03-13T07:36:17.483 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:12 9.1 9:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) v9 -- 0x55a2c1848130 con 0x55a2c16c5ed0 2024-03-13T07:36:17.483 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:17.483 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 5 client.4373.objecter 1 in flight 2024-03-13T07:36:17.483 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c18484d0 2024-03-13T07:36:17.483 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 41 base noexist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:17.483 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target noexist @9 -> pgid 9.a7bbaa1 2024-03-13T07:36:17.484 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.a7bbaa1 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:17.484 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:17.484 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid noexist '@9' '@9' [read 0~0] tid 13 osd.7 2024-03-13T07:36:17.484 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:17.484 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 13 2024-03-13T07:36:17.484 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 15 client.4373.objecter _send_op 13 to 9.1 on osd.7 2024-03-13T07:36:17.484 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:13 9.1 9:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) v9 -- 0x55a2c186c1b0 con 0x55a2c16c5ed0 2024-03-13T07:36:17.484 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 5 2024-03-13T07:36:17.484 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.002+0000 7f71e0bf5780 5 client.4373.objecter 2 in flight 2024-03-13T07:36:17.484 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 16 ==== osd_map(42..42 src has 1..42) v4 ==== 1385+0+0 (crc 0 0 0) 0x7f71d4013ad0 con 0x55a2c16c5ed0 2024-03-13T07:36:17.484 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 17 ==== osd_op_reply(12 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:17.484 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71cf7fe640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_map(42..42 src has 1..42) v4 2024-03-13T07:36:17.485 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(12 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 2024-03-13T07:36:17.485 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:17.540 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map got epochs [42,42] > 41 2024-03-13T07:36:17.540 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 42 2024-03-13T07:36:17.540 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71cf7fe640 10 client.4373.objecter checking op 12 2024-03-13T07:36:17.540 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 42 base exist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:17.541 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:17.541 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71cf7fe640 10 client.4373.objecter checking op 13 2024-03-13T07:36:17.541 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 42 base noexist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:17.541 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target noexist @9 -> pgid 9.a7bbaa1 2024-03-13T07:36:17.541 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71cf7fe640 20 client.4373.objecter dump_active .. 0 homeless 2024-03-13T07:36:17.541 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71cf7fe640 20 client.4373.objecter 12 9.d3a59201 osd.7 exist [read 0~0] 2024-03-13T07:36:17.541 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71cf7fe640 20 client.4373.objecter 13 9.a7bbaa1 osd.7 noexist [read 0~0] 2024-03-13T07:36:17.541 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 12 ondisk uv 4 in 9.1 attempt 0 2024-03-13T07:36:17.541 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 5 2024-03-13T07:36:17.541 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 12 2024-03-13T07:36:17.541 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71ddef4640 15 client.4373.objecter _finish_op 12 2024-03-13T07:36:17.541 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 5 2024-03-13T07:36:17.541 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 12 2024-03-13T07:36:17.541 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.483+0000 7f71ddef4640 5 client.4373.objecter 1 in flight 2024-03-13T07:36:17.542 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 18 ==== osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:17.542 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2024-03-13T07:36:17.542 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:17.542 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 13 ondisk uv 0 in 9.1 attempt 0 2024-03-13T07:36:17.542 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71ddef4640 10 client.4373.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-03-13T07:36:17.542 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 0 2024-03-13T07:36:17.542 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 13 2024-03-13T07:36:17.542 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71ddef4640 15 client.4373.objecter _finish_op 13 2024-03-13T07:36:17.542 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:17.542 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 13 2024-03-13T07:36:17.542 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71ddef4640 5 client.4373.objecter 0 in flight 2024-03-13T07:36:17.543 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (538 ms) 2024-03-13T07:36:17.793 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-03-13T07:36:17.793 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c1847d90 2024-03-13T07:36:17.793 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 42 base exist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:17.793 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:17.793 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.d3a59201 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:17.793 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:17.793 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid exist '@9' '@9' [read 0~0] tid 14 osd.7 2024-03-13T07:36:17.793 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:17.793 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 14 2024-03-13T07:36:17.793 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 15 client.4373.objecter _send_op 14 to 9.1 on osd.7 2024-03-13T07:36:17.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:14 9.1 9:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v9 -- 0x55a2c1848240 con 0x55a2c16c5ed0 2024-03-13T07:36:17.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:17.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 5 client.4373.objecter 1 in flight 2024-03-13T07:36:17.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c18487c0 2024-03-13T07:36:17.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 42 base noexist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:17.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target noexist @9 -> pgid 9.a7bbaa1 2024-03-13T07:36:17.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.a7bbaa1 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:17.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:17.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid noexist '@9' '@9' [read 0~0] tid 15 osd.7 2024-03-13T07:36:17.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:17.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 15 2024-03-13T07:36:17.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 15 client.4373.objecter _send_op 15 to 9.1 on osd.7 2024-03-13T07:36:17.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:15 9.1 9:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v9 -- 0x55a2c186c1b0 con 0x55a2c16c5ed0 2024-03-13T07:36:17.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 5 2024-03-13T07:36:17.794 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.540+0000 7f71e0bf5780 5 client.4373.objecter 2 in flight 2024-03-13T07:36:17.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (252 ms) 2024-03-13T07:36:17.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-03-13T07:36:17.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 19 ==== osd_op_reply(14 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:17.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(14 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 2024-03-13T07:36:17.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:17.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 14 ondisk uv 4 in 9.1 attempt 0 2024-03-13T07:36:17.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 5 2024-03-13T07:36:17.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 14 2024-03-13T07:36:17.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 15 client.4373.objecter _finish_op 14 2024-03-13T07:36:17.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 5 2024-03-13T07:36:17.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 14 2024-03-13T07:36:17.795 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 5 client.4373.objecter 1 in flight 2024-03-13T07:36:17.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 20 ==== osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:17.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2024-03-13T07:36:17.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:17.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 15 ondisk uv 0 in 9.1 attempt 0 2024-03-13T07:36:17.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 10 client.4373.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-03-13T07:36:17.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 0 2024-03-13T07:36:17.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 15 2024-03-13T07:36:17.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 15 client.4373.objecter _finish_op 15 2024-03-13T07:36:17.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:17.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 15 2024-03-13T07:36:17.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.792+0000 7f71ddef4640 5 client.4373.objecter 0 in flight 2024-03-13T07:36:17.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c18487c0 2024-03-13T07:36:17.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 42 base exist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:17.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:17.796 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.d3a59201 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:17.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:17.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid exist '@9' '@9' [read 0~0] tid 16 osd.7 2024-03-13T07:36:17.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:17.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 16 2024-03-13T07:36:17.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 15 client.4373.objecter _send_op 16 to 9.1 on osd.7 2024-03-13T07:36:17.916 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:16 9.1 9:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v9 -- 0x55a2c186b210 con 0x55a2c16c5ed0 2024-03-13T07:36:17.917 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:17.917 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 5 client.4373.objecter 1 in flight 2024-03-13T07:36:17.917 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c186b710 2024-03-13T07:36:17.917 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 42 base noexist @9 precalc_pgid 0 pgid 0.0 is_read 2024-03-13T07:36:17.917 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target noexist @9 -> pgid 9.a7bbaa1 2024-03-13T07:36:17.917 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.a7bbaa1 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:17.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:17.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid noexist '@9' '@9' [read 0~0] tid 17 osd.7 2024-03-13T07:36:17.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:17.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 17 2024-03-13T07:36:17.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 15 client.4373.objecter _send_op 17 to 9.1 on osd.7 2024-03-13T07:36:17.918 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:17 9.1 9:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v9 -- 0x55a2c186bbc0 con 0x55a2c16c5ed0 2024-03-13T07:36:17.919 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 5 2024-03-13T07:36:17.919 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.793+0000 7f71e0bf5780 5 client.4373.objecter 2 in flight 2024-03-13T07:36:17.919 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.795+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:17.919 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.795+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:17.919 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.795+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:17.919 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 21 ==== osd_op_reply(16 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:17.920 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(16 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 2024-03-13T07:36:17.920 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:17.920 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 16 ondisk uv 4 in 9.1 attempt 0 2024-03-13T07:36:17.920 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 5 2024-03-13T07:36:17.920 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 16 2024-03-13T07:36:17.920 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 15 client.4373.objecter _finish_op 16 2024-03-13T07:36:17.920 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 5 2024-03-13T07:36:17.921 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 16 2024-03-13T07:36:17.921 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 5 client.4373.objecter 1 in flight 2024-03-13T07:36:18.963 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 22 ==== osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:18.963 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2024-03-13T07:36:18.963 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:18.963 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 17 ondisk uv 0 in 9.1 attempt 0 2024-03-13T07:36:18.963 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 10 client.4373.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-03-13T07:36:18.963 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 0 2024-03-13T07:36:18.963 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 17 2024-03-13T07:36:18.963 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 15 client.4373.objecter _finish_op 17 2024-03-13T07:36:18.963 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:18.963 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 17 2024-03-13T07:36:18.964 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71ddef4640 5 client.4373.objecter 0 in flight 2024-03-13T07:36:18.964 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (124 ms) 2024-03-13T07:36:18.964 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-03-13T07:36:18.964 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71e0bf5780 20 librados: queue_aio_write 0x55a2c1637a70 completion 0x55a2c18486b0 write_seq 4 2024-03-13T07:36:18.964 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c18487c0 2024-03-13T07:36:18.964 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 42 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:18.964 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:18.964 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.d3a59201 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:18.964 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:18.964 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid exist '@9' '@9' [writefull 0~5 in=5b] tid 18 osd.7 2024-03-13T07:36:18.964 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:18.964 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 18 2024-03-13T07:36:18.964 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71e0bf5780 15 client.4373.objecter _send_op 18 to 9.1 on osd.7 2024-03-13T07:36:18.964 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:18 9.1 9:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v9 -- 0x55a2c1847d90 con 0x55a2c16c5ed0 2024-03-13T07:36:18.964 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:18.965 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:17.916+0000 7f71e0bf5780 5 client.4373.objecter 1 in flight 2024-03-13T07:36:18.965 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.795+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:18.965 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.795+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:18.965 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.795+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:19.327 INFO:tasks.workunit.client.0.smithi132.stdout: pool: Running main() from gmock_main.cc 2024-03-13T07:36:19.327 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-03-13T07:36:19.327 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [----------] Global test environment set-up. 2024-03-13T07:36:19.327 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-03-13T07:36:19.327 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-03-13T07:36:19.327 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [ OK ] NeoRadosPools.PoolList (4401 ms) 2024-03-13T07:36:19.327 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-03-13T07:36:19.327 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (2918 ms) 2024-03-13T07:36:19.327 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-03-13T07:36:19.327 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (4547 ms) 2024-03-13T07:36:19.327 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-03-13T07:36:19.327 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (4817 ms) 2024-03-13T07:36:19.328 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-03-13T07:36:19.328 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (2534 ms) 2024-03-13T07:36:19.328 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-03-13T07:36:19.328 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (7932 ms) 2024-03-13T07:36:19.328 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [----------] 6 tests from NeoRadosPools (27149 ms total) 2024-03-13T07:36:19.328 INFO:tasks.workunit.client.0.smithi132.stdout: pool: 2024-03-13T07:36:19.328 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [----------] Global test environment tear-down 2024-03-13T07:36:19.328 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [==========] 6 tests from 1 test suite ran. (27149 ms total) 2024-03-13T07:36:19.328 INFO:tasks.workunit.client.0.smithi132.stdout: pool: [ PASSED ] 6 tests. 2024-03-13T07:36:19.406 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 23 ==== osd_op_reply(18 exist [writefull 0~5] v42'5 uv5 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:19.406 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(18 exist [writefull 0~5] v42'5 uv5 ondisk = 0) v8 2024-03-13T07:36:19.406 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:19.406 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 18 ondisk uv 5 in 9.1 attempt 0 2024-03-13T07:36:19.406 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 0 2024-03-13T07:36:19.406 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 18 2024-03-13T07:36:19.406 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71ddef4640 15 client.4373.objecter _finish_op 18 2024-03-13T07:36:19.406 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:19.406 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 18 2024-03-13T07:36:19.407 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71ddef4640 5 client.4373.objecter 0 in flight 2024-03-13T07:36:19.407 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71ddef4640 20 librados: complete_aio_write 0x55a2c18486b0 2024-03-13T07:36:19.407 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (1047 ms) 2024-03-13T07:36:19.407 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-03-13T07:36:19.407 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71e0bf5780 20 librados: queue_aio_write 0x55a2c1637a70 completion 0x55a2c18476b0 write_seq 5 2024-03-13T07:36:19.407 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c18487c0 2024-03-13T07:36:19.407 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 42 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:19.407 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:19.407 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.d3a59201 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:19.407 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71e0bf5780 20 client.4373.objecter _get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:19.407 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid exist '@9' '@9' [writefull 0~5 in=5b] tid 19 osd.7 2024-03-13T07:36:19.407 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:19.407 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 19 2024-03-13T07:36:19.408 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71e0bf5780 15 client.4373.objecter _send_op 19 to 9.1 on osd.7 2024-03-13T07:36:19.408 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:19 9.1 9:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v9 -- 0x55a2c1847d90 con 0x55a2c16c5ed0 2024-03-13T07:36:19.408 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:19.408 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:18.963+0000 7f71e0bf5780 5 client.4373.objecter 1 in flight 2024-03-13T07:36:19.408 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:19.406+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 24 ==== osd_map(43..43 src has 1..43) v4 ==== 312+0+0 (crc 0 0 0) 0x7f71d4005530 con 0x55a2c16c5ed0 2024-03-13T07:36:19.408 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:19.406+0000 7f71cf7fe640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_map(43..43 src has 1..43) v4 2024-03-13T07:36:21.432 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:19.406+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map got epochs [43,43] > 42 2024-03-13T07:36:21.432 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:19.406+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 43 2024-03-13T07:36:21.432 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:19.406+0000 7f71cf7fe640 10 client.4373.objecter checking op 19 2024-03-13T07:36:21.432 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:19.406+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 43 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:21.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:19.406+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:21.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:19.406+0000 7f71cf7fe640 20 client.4373.objecter dump_active .. 0 homeless 2024-03-13T07:36:21.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:19.406+0000 7f71cf7fe640 20 client.4373.objecter 19 9.d3a59201 osd.7 exist [writefull 0~5 in=5b] 2024-03-13T07:36:21.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:19.795+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:21.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:19.795+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:21.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:19.795+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:21.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:20.795+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:21.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:20.795+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:21.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:20.795+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:21.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 25 ==== osd_op_reply(19 exist [writefull 0~5] v43'6 uv6 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:21.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(19 exist [writefull 0~5] v43'6 uv6 ondisk = 0) v8 2024-03-13T07:36:21.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:21.433 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 19 ondisk uv 6 in 9.1 attempt 0 2024-03-13T07:36:21.434 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 0 2024-03-13T07:36:21.434 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 19 2024-03-13T07:36:21.434 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71ddef4640 15 client.4373.objecter _finish_op 19 2024-03-13T07:36:21.434 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:21.434 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 19 2024-03-13T07:36:21.434 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71ddef4640 5 client.4373.objecter 0 in flight 2024-03-13T07:36:21.434 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71ddef4640 20 librados: complete_aio_write 0x55a2c18476b0 2024-03-13T07:36:21.434 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (2469 ms) 2024-03-13T07:36:21.434 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-03-13T07:36:21.434 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71e0bf5780 20 librados: queue_aio_write 0x55a2c1637a70 completion 0x55a2c1848230 write_seq 6 2024-03-13T07:36:21.434 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71e0bf5780 10 client.4373.objecter _op_submit op 0x55a2c18487c0 2024-03-13T07:36:21.434 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71e0bf5780 20 client.4373.objecter _calc_target epoch 43 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:21.434 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71e0bf5780 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:21.434 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71e0bf5780 10 client.4373.objecter _calc_target raw pgid 9.d3a59201 -> actual 9.1 acting [7,4] primary 7 2024-03-13T07:36:21.632 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71e0bf5780 20 client.4373.objecter _ api_service: [==========] Running 4 tests from 1 test suite. 2024-03-13T07:36:21.632 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: [----------] Global test environment set-up. 2024-03-13T07:36:21.632 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: [----------] 4 tests from LibRadosService 2024-03-13T07:36:21.632 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-03-13T07:36:21.632 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5389 ms) 2024-03-13T07:36:21.632 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-03-13T07:36:21.633 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: [ OK ] LibRadosService.RegisterLate (638 ms) 2024-03-13T07:36:21.633 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-03-13T07:36:21.633 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: cluster: 2024-03-13T07:36:21.633 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: id: 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:36:21.633 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: health: HEALTH_WARN 2024-03-13T07:36:21.633 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 20 pool(s) do not have an application enabled 2024-03-13T07:36:21.633 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 2024-03-13T07:36:21.633 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: services: 2024-03-13T07:36:21.633 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 2m) 2024-03-13T07:36:21.633 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-03-13T07:36:21.633 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: osd: 8 osds: 8 up (since 74s), 8 in (since 2m) 2024-03-13T07:36:21.633 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-03-13T07:36:21.633 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 2024-03-13T07:36:21.633 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: data: 2024-03-13T07:36:21.633 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: pools: 36 pools, 1097 pgs 2024-03-13T07:36:21.634 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: objects: 2 objects, 449 KiB 2024-03-13T07:36:21.634 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: usage: 214 MiB used, 720 GiB / 720 GiB avail 2024-03-13T07:36:21.634 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: pgs: 93.254% pgs unknown 2024-03-13T07:36:21.634 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 5.925% pgs not active 2024-03-13T07:36:21.634 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 1023 unknown 2024-03-13T07:36:21.634 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 35 creating+activating 2024-03-13T07:36:21.634 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 30 creating+peering 2024-03-13T07:36:21.634 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 9 active+clean 2024-03-13T07:36:21.634 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 2024-03-13T07:36:21.634 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 2024-03-13T07:36:21.634 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: cluster: 2024-03-13T07:36:21.634 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: id: 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T07:36:21.634 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: health: HEALTH_WARN 2024-03-13T07:36:21.635 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 19 pool(s) do not have an application enabled 2024-03-13T07:36:21.635 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 2024-03-13T07:36:21.635 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: services: 2024-03-13T07:36:21.635 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 2m) 2024-03-13T07:36:21.635 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-03-13T07:36:21.635 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: osd: 8 osds: 8 up (since 76s), 8 in (since 2m) 2024-03-13T07:36:21.635 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-03-13T07:36:21.635 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-03-13T07:36:21.635 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 2024-03-13T07:36:21.635 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: data: 2024-03-13T07:36:21.635 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: pools: 39 pools, 1121 pgs 2024-03-13T07:36:21.635 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: objects: 2 objects, 449 KiB 2024-03-13T07:36:21.635 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: usage: 214 MiB used, 720 GiB / 720 GiB avail 2024-03-13T07:36:21.635 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: pgs: 72.168% pgs unknown 2024-03-13T07:36:21.635 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 26.940% pgs not active 2024-03-13T07:36:21.636 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 809 unknown 2024-03-13T07:36:21.636 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 212 creating+peering 2024-03-13T07:36:21.636 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 90 creating+activating 2024-03-13T07:36:21.636 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 10 active+clean 2024-03-13T07:36:21.636 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 2024-03-13T07:36:21.636 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 2024-03-13T07:36:21.636 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: [ OK ] LibRadosService.StatusFormat (3315 ms) 2024-03-13T07:36:21.636 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: [ RUN ] LibRadosService.Status 2024-03-13T07:36:21.636 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: [ OK ] LibRadosService.Status (20333 ms) 2024-03-13T07:36:21.636 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: [----------] 4 tests from LibRadosService (29675 ms total) 2024-03-13T07:36:21.636 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: 2024-03-13T07:36:21.636 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: [----------] Global test environment tear-down 2024-03-13T07:36:21.636 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: [==========] 4 tests from 1 test suite ran. (29675 ms total) 2024-03-13T07:36:21.636 INFO:tasks.workunit.client.0.smithi132.stdout: api_service: [ PASSED ] 4 tests. 2024-03-13T07:36:22.176 INFO:tasks.workunit.client.0.smithi132.stdout:get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:22.176 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71e0bf5780 10 client.4373.objecter _op_submit oid exist '@9' '@9' [writefull 0~5 in=5b] tid 20 osd.7 2024-03-13T07:36:22.176 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71e0bf5780 20 client.4373.objecter get_session s=0x55a2c183aaf0 osd=7 3 2024-03-13T07:36:22.176 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71e0bf5780 15 client.4373.objecter _session_op_assign 7 20 2024-03-13T07:36:22.176 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71e0bf5780 15 client.4373.objecter _send_op 20 to 9.1 on osd.7 2024-03-13T07:36:22.177 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- osd_op(unknown.0.0:20 9.1 9:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v9 -- 0x55a2c1847d90 con 0x55a2c16c5ed0 2024-03-13T07:36:22.177 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:22.177 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.432+0000 7f71e0bf5780 5 client.4373.objecter 1 in flight 2024-03-13T07:36:22.177 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.461+0000 7f71cf7fe640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 26 ==== osd_map(44..44 src has 1..44) v4 ==== 1074+0+0 (crc 0 0 0) 0x7f71d4013ad0 con 0x55a2c16c5ed0 2024-03-13T07:36:22.177 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.461+0000 7f71cf7fe640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_map(44..44 src has 1..44) v4 2024-03-13T07:36:22.177 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.461+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map got epochs [44,44] > 43 2024-03-13T07:36:22.177 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.461+0000 7f71cf7fe640 3 client.4373.objecter handle_osd_map decoding incremental epoch 44 2024-03-13T07:36:22.177 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.461+0000 7f71cf7fe640 10 client.4373.objecter checking op 20 2024-03-13T07:36:22.177 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.461+0000 7f71cf7fe640 20 client.4373.objecter _calc_target epoch 44 base exist @9 precalc_pgid 0 pgid 0.0 is_write 2024-03-13T07:36:22.177 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.461+0000 7f71cf7fe640 20 client.4373.objecter _calc_target target exist @9 -> pgid 9.d3a59201 2024-03-13T07:36:22.177 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.461+0000 7f71cf7fe640 20 client.4373.objecter dump_active .. 0 homeless 2024-03-13T07:36:22.177 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.461+0000 7f71cf7fe640 20 client.4373.objecter 20 9.d3a59201 osd.7 exist [writefull 0~5 in=5b] 2024-03-13T07:36:22.177 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.795+0000 7f71ce7fc640 10 monclient: tick 2024-03-13T07:36:22.177 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.795+0000 7f71ce7fc640 10 monclient: _check_auth_tickets 2024-03-13T07:36:22.177 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.795+0000 7f71ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T07:36:22.178 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:21.850+0000 7f71e0959640 10 client.4373.objecter tick 2024-03-13T07:36:22.178 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71ddef4640 1 -- 172.21.15.132:0/2636667481 <== osd.7 v2:172.21.15.195:6820/3378300314 27 ==== osd_op_reply(20 exist [writefull 0~5] v44'7 uv7 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f71d4013440 con 0x55a2c16c5ed0 2024-03-13T07:36:22.178 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71ddef4640 10 client.4373.objecter ms_dispatch 0x55a2c166ed10 osd_op_reply(20 exist [writefull 0~5] v44'7 uv7 ondisk = 0) v8 2024-03-13T07:36:22.178 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71ddef4640 10 client.4373.objecter in handle_osd_op_reply 2024-03-13T07:36:22.178 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71ddef4640 7 client.4373.objecter handle_osd_op_reply 20 ondisk uv 7 in 9.1 attempt 0 2024-03-13T07:36:22.178 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71ddef4640 10 client.4373.objecter op 0 rval 0 len 0 2024-03-13T07:36:22.178 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71ddef4640 15 client.4373.objecter handle_osd_op_reply completed tid 20 2024-03-13T07:36:22.178 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71ddef4640 15 client.4373.objecter _finish_op 20 2024-03-13T07:36:22.178 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71ddef4640 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 4 2024-03-13T07:36:22.179 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71ddef4640 15 client.4373.objecter _session_op_remove 7 20 2024-03-13T07:36:22.179 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71ddef4640 5 client.4373.objecter 0 in flight 2024-03-13T07:36:22.179 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71ddef4640 20 librados: complete_aio_write 0x55a2c1848230 2024-03-13T07:36:22.179 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (744 ms) 2024-03-13T07:36:22.179 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71e0bf5780 10 librados: watch_flush enter 2024-03-13T07:36:22.179 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71e0bf5780 10 librados: watch_flush exit 2024-03-13T07:36:22.179 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71e0bf5780 10 client.4373.objecter close_session for osd.7 2024-03-13T07:36:22.179 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 >> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] conn(0x55a2c16c5ed0 msgr2=0x55a2c1846ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:36:22.179 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71e0bf5780 1 --2- 172.21.15.132:0/2636667481 >> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] conn(0x55a2c16c5ed0 0x55a2c1846ef0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T07:36:22.179 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71e0bf5780 20 client.4373.objecter put_session s=0x55a2c183aaf0 osd=7 1 2024-03-13T07:36:22.179 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71e0bf5780 20 client.4373.objecter shutdown clearing up homeless session... 2024-03-13T07:36:22.179 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.176+0000 7f71e0bf5780 10 client.4373.objecter successfully canceled tick 2024-03-13T07:36:22.179 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.177+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f71c006b7d0 msgr2=0x7f71c006dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:36:22.179 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.177+0000 7f71e0bf5780 1 --2- 172.21.15.132:0/2636667481 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f71c006b7d0 0x7f71c006dc90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f71d0054000 tx=0x7f71d004c000 comp rx=0 tx=0).stop 2024-03-13T07:36:22.179 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.177+0000 7f71e0bf5780 10 monclient: shutdown 2024-03-13T07:36:22.180 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.177+0000 7f71e0bf5780 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T07:36:22.180 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.177+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2c179ada0 msgr2=0x55a2c16c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T07:36:22.180 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.177+0000 7f71e0bf5780 1 --2- 172.21.15.132:0/2636667481 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2c179ada0 0x55a2c16c7580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005e80 tx=0x7f71d8008430 comp rx=0 tx=0).stop 2024-03-13T07:36:22.180 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.177+0000 7f71deef6640 1 -- 172.21.15.132:0/2636667481 reap_dead start 2024-03-13T07:36:22.180 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.177+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 shutdown_connections 2024-03-13T07:36:22.180 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.177+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 >> 172.21.15.132:0/2636667481 conn(0x55a2c16c0290 msgr2=0x55a2c17991c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T07:36:22.180 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.177+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 shutdown_connections 2024-03-13T07:36:22.180 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.178+0000 7f71e0bf5780 1 -- 172.21.15.132:0/2636667481 wait complete. 2024-03-13T07:36:22.180 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.178+0000 7f71e0bf5780 1 librados: shutdown 2024-03-13T07:36:22.180 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [----------] 12 tests from AsioRados (19581 ms total) 2024-03-13T07:36:22.180 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: 2024-03-13T07:36:22.180 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [----------] Global test environment tear-down 2024-03-13T07:36:22.182 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (30432 ms total) 2024-03-13T07:36:22.182 INFO:tasks.workunit.client.0.smithi132.stdout: api_asio: [ PASSED ] 12 tests. 2024-03-13T07:36:23.666 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-03-13T07:36:23.666 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: [----------] Global test environment set-up. 2024-03-13T07:36:23.666 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-03-13T07:36:23.666 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-03-13T07:36:23.666 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5421 ms) 2024-03-13T07:36:23.666 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-03-13T07:36:23.666 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (572 ms) 2024-03-13T07:36:23.666 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-03-13T07:36:23.666 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20495 ms) 2024-03-13T07:36:23.666 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-03-13T07:36:23.666 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: attempt 0 of 20 2024-03-13T07:36:23.667 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: failed to deregister: 2024-03-13T07:36:23.667 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: { 2024-03-13T07:36:23.667 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "epoch": 12, 2024-03-13T07:36:23.667 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "modified": "2024-03-13T07:36:19.092018+0000", 2024-03-13T07:36:23.667 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "services": { 2024-03-13T07:36:23.667 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "laundry": { 2024-03-13T07:36:23.667 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "daemons": { 2024-03-13T07:36:23.667 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "summary": "", 2024-03-13T07:36:23.667 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "close-test-pid41476": { 2024-03-13T07:36:23.667 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "start_epoch": 12, 2024-03-13T07:36:23.667 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "start_stamp": "2024-03-13T07:36:18.682450+0000", 2024-03-13T07:36:23.667 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "gid": 4563, 2024-03-13T07:36:23.667 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "addr": "172.21.15.132:0/591969771", 2024-03-13T07:36:23.667 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "metadata": { 2024-03-13T07:36:23.667 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "arch": "x86_64", 2024-03-13T07:36:23.668 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "ceph_release": "squid", 2024-03-13T07:36:23.668 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-2007-gb7c579c9 (b7c579c9c3ed531070a6aa96f74f80b53f80de32) squid (dev)", 2024-03-13T07:36:23.668 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "ceph_version_short": "19.0.0-2007-gb7c579c9", 2024-03-13T07:36:23.668 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-03-13T07:36:23.668 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "distro": "centos", 2024-03-13T07:36:23.668 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-03-13T07:36:23.668 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "distro_version": "9", 2024-03-13T07:36:23.668 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "foo": "bar", 2024-03-13T07:36:23.668 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "hostname": "smithi132", 2024-03-13T07:36:23.668 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024", 2024-03-13T07:36:23.668 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "kernel_version": "5.14.0-427.el9.x86_64", 2024-03-13T07:36:23.668 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "mem_swap_kb": "0", 2024-03-13T07:36:23.668 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "mem_total_kb": "32488480", 2024-03-13T07:36:23.668 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "os": "Linux", 2024-03-13T07:36:23.668 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "this": "that" 2024-03-13T07:36:23.669 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: }, 2024-03-13T07:36:23.669 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "task_status": {} 2024-03-13T07:36:23.669 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: }, 2024-03-13T07:36:23.669 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "pid41441": { 2024-03-13T07:36:23.669 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "start_epoch": 11, 2024-03-13T07:36:23.669 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "start_stamp": "2024-03-13T07:36:01.649787+0000", 2024-03-13T07:36:23.669 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "gid": 4684, 2024-03-13T07:36:23.669 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "addr": "172.21.15.132:0/344162118", 2024-03-13T07:36:23.669 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "metadata": { 2024-03-13T07:36:23.669 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "arch": "x86_64", 2024-03-13T07:36:23.669 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "ceph_release": "squid", 2024-03-13T07:36:23.669 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-2007-gb7c579c9 (b7c579c9c3ed531070a6aa96f74f80b53f80de32) squid (dev)", 2024-03-13T07:36:23.669 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "ceph_version_short": "19.0.0-2007-gb7c579c9", 2024-03-13T07:36:23.669 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-03-13T07:36:26.356 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-03-13T07:36:26.333+0000 ea52640 -1 osd.6 47 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4463.0:2 31.1a 31.db6ee63a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e28) 2024-03-13T07:36:27.334 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-03-13T07:36:27.327+0000 ea52640 -1 osd.6 47 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4463.0:2 31.1a 31:5c6776db:::0.obj:head [write 0~239 in=239b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) 2024-03-13T07:36:28.617 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T07:36:28.580+0000 12a5a640 -1 osd.4 48 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4387.0:2 4.0 4.1e50c260 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e28) 2024-03-13T07:36:28.643 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-03-13T07:36:28.575+0000 ea52640 -1 osd.6 48 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4463.0:2 31.1a 31:5c6776db:::0.obj:head [write 0~239 in=239b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) 2024-03-13T07:36:29.617 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T07:36:29.601+0000 12a5a640 -1 osd.4 48 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4387.0:2 4.0 4:06430a78:test-rados-api-smithi132-40865-3::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) 2024-03-13T07:36:29.735 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-03-13T07:36:29.632+0000 ea52640 -1 osd.6 48 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4463.0:2 31.1a 31:5c6776db:::0.obj:head [write 0~239 in=239b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) 2024-03-13T07:36:30.592 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T07:36:30.589+0000 12a5a640 -1 osd.4 48 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4387.0:2 4.0 4:06430a78:test-rados-api-smithi132-40865-3::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) 2024-03-13T07:36:31.985 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T07:36:31.862+0000 12a5a640 -1 osd.4 49 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4387.0:2 4.0 4:06430a78:test-rados-api-smithi132-40865-3::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) 2024-03-13T07:36:33.275 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T07:36:33.111+0000 12a5a640 -1 osd.4 49 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4387.0:2 4.0 4:06430a78:test-rados-api-smithi132-40865-3::foo:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) 2024-03-13T07:36:34.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "distro": "centos", 2024-03-13T07:36:34.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-03-13T07:36:34.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "distro_version": "9", 2024-03-13T07:36:34.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "foo": "bar", 2024-03-13T07:36:34.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "hostname": "smithi132", 2024-03-13T07:36:34.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024", 2024-03-13T07:36:34.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "kernel_version": "5.14.0-427.el9.x86_64", 2024-03-13T07:36:34.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "mem_swap_kb": "0", 2024-03-13T07:36:34.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "mem_total_kb": "32488480", 2024-03-13T07:36:34.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "os": "Linux", 2024-03-13T07:36:34.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "this": "that" 2024-03-13T07:36:34.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: }, 2024-03-13T07:36:34.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "task_status": {} 2024-03-13T07:36:34.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: } 2024-03-13T07:36:34.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: } 2024-03-13T07:36:34.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: } 2024-03-13T07:36:34.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: } 2024-03-13T07:36:34.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: } 2024-03-13T07:36:34.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: 2024-03-13T07:36:34.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: attempt 1 of 20 2024-03-13T07:36:34.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: failed to deregister: 2024-03-13T07:36:34.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: { 2024-03-13T07:36:34.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "epoch": 14, 2024-03-13T07:36:34.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "modified": "2024-03-13T07:36:25.096249+0000", 2024-03-13T07:36:34.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "services": { 2024-03-13T07:36:34.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "laundry": { 2024-03-13T07:36:34.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "daemons": { 2024-03-13T07:36:34.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "summary": "", 2024-03-13T07:36:34.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "close-test-pid41476": { 2024-03-13T07:36:34.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "start_epoch": 14, 2024-03-13T07:36:34.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "start_stamp": "2024-03-13T07:36:24.112682+0000", 2024-03-13T07:36:34.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "gid": 4747, 2024-03-13T07:36:34.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "addr": "172.21.15.132:0/1071102455", 2024-03-13T07:36:34.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "metadata": { 2024-03-13T07:36:34.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "arch": "x86_64", 2024-03-13T07:36:34.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "ceph_release": "squid", 2024-03-13T07:36:34.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-2007-gb7c579c9 (b7c579c9c3ed531070a6aa96f74f80b53f80de32) squid (dev)", 2024-03-13T07:36:34.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "ceph_version_short": "19.0.0-2007-gb7c579c9", 2024-03-13T07:36:34.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-03-13T07:36:34.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "distro": "centos", 2024-03-13T07:36:34.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-03-13T07:36:34.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "distro_version": "9", 2024-03-13T07:36:34.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "foo": "bar", 2024-03-13T07:36:34.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "hostname": "smithi132", 2024-03-13T07:36:34.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Fri Feb 23 04:45:07 UTC 2024", 2024-03-13T07:36:34.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "kernel_version": "5.14.0-427.el9.x86_64", 2024-03-13T07:36:34.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "mem_swap_kb": "0", 2024-03-13T07:36:34.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "mem_total_kb": "32488480", 2024-03-13T07:36:34.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "os": "Linux", 2024-03-13T07:36:34.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "this": "that" 2024-03-13T07:36:34.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: }, 2024-03-13T07:36:34.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: "task_status": {} 2024-03-13T07:36:34.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: } 2024-03-13T07:36:34.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: } 2024-03-13T07:36:34.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: } 2024-03-13T07:36:34.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: } 2024-03-13T07:36:34.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: } 2024-03-13T07:36:34.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: 2024-03-13T07:36:34.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: attempt 2 of 20 2024-03-13T07:36:34.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (16087 ms) 2024-03-13T07:36:34.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (42575 ms total) 2024-03-13T07:36:34.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: 2024-03-13T07:36:34.552 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: [----------] Global test environment tear-down 2024-03-13T07:36:34.552 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (42575 ms total) 2024-03-13T07:36:34.552 INFO:tasks.workunit.client.0.smithi132.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-03-13T07:36:34.618 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T07:36:34.541+0000 12a5a640 -1 osd.4 50 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4516.0:1 39.6s0 39.7fc1f406 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e30) 2024-03-13T07:36:35.696 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T07:36:35.688+0000 12a5a640 -1 osd.4 50 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4516.0:1 39.6s0 39:602f83fe:::foo:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) 2024-03-13T07:36:36.937 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T07:36:36.927+0000 12a5a640 -1 osd.4 51 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4516.0:1 39.6s0 39:602f83fe:::foo:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) 2024-03-13T07:36:39.679 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:03:03.569 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:39.757 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:03:03.648 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:39.816 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:03:03.706 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:39.821 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:03:03.711 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:40.006 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:03:03.897 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:40.109 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:03:03.979 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:40.138 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:03:04.009 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:40.332 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:03:04.222 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:40.357 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:03:04.227 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:36:45.954 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-03-13T07:36:45.954 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [----------] Global test environment set-up. 2024-03-13T07:36:45.954 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-03-13T07:36:45.954 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-03-13T07:36:45.954 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-03-13T07:36:45.954 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-13T07:36:45.954 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: 2024-03-13T07:36:45.955 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-03-13T07:36:45.955 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-03-13T07:36:45.955 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-03-13T07:36:45.955 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: 2024-03-13T07:35:51.987+0000 7fe29e20e780 -1 failed for service _ceph-mon._tcp 2024-03-13T07:36:45.955 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: 2024-03-13T07:35:51.987+0000 7fe29e20e780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-03-13T07:36:45.955 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (230 ms) 2024-03-13T07:36:45.955 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-03-13T07:36:45.955 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2030 ms) 2024-03-13T07:36:45.955 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2260 ms total) 2024-03-13T07:36:45.955 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: 2024-03-13T07:36:45.955 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-03-13T07:36:45.955 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-03-13T07:36:45.955 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092700de50 2024-03-13T07:36:45.955 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092648b5b0 2024-03-13T07:36:45.956 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x5609263db7b0 2024-03-13T07:36:45.956 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926570fe0 2024-03-13T07:36:45.956 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092676b0e0 2024-03-13T07:36:45.956 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092676bb10 2024-03-13T07:36:45.956 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092676c520 2024-03-13T07:36:45.956 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092676cfa0 2024-03-13T07:36:45.956 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092676d9f0 2024-03-13T07:36:45.956 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926761150 2024-03-13T07:36:45.956 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x5609266be4c0 2024-03-13T07:36:45.956 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092676ea30 2024-03-13T07:36:45.956 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092676f4d0 2024-03-13T07:36:45.956 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092676d780 2024-03-13T07:36:45.956 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926770360 2024-03-13T07:36:45.956 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092676f210 2024-03-13T07:36:45.957 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926770e70 2024-03-13T07:36:45.957 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926771900 2024-03-13T07:36:45.957 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x5609267700f0 2024-03-13T07:36:45.957 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926772820 2024-03-13T07:36:45.957 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926771690 2024-03-13T07:36:45.957 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926773700 2024-03-13T07:36:45.957 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926772580 2024-03-13T07:36:45.957 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926534180 2024-03-13T07:36:45.957 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x5609267734a0 2024-03-13T07:36:45.957 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926535080 2024-03-13T07:36:45.957 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926533ef0 2024-03-13T07:36:45.957 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926535bc0 2024-03-13T07:36:45.957 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926536630 2024-03-13T07:36:45.958 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926534e30 2024-03-13T07:36:45.958 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926537520 2024-03-13T07:36:45.958 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x5609265363f0 2024-03-13T07:36:45.958 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926538490 2024-03-13T07:36:45.958 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x5609265372c0 2024-03-13T07:36:45.958 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926539360 2024-03-13T07:36:45.958 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x5609265381a0 2024-03-13T07:36:45.958 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092653a280 2024-03-13T07:36:45.958 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x5609265390b0 2024-03-13T07:36:45.958 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092653adc0 2024-03-13T07:36:45.958 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092653b850 2024-03-13T07:36:45.958 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x560926539fe0 2024-03-13T07:36:45.958 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092653c740 2024-03-13T07:36:45.958 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092653b580 2024-03-13T07:36:45.958 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092653d280 2024-03-13T07:36:45.959 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092653dc80 2024-03-13T07:36:45.959 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092676ffe0 2024-03-13T07:36:45.959 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092676d890 2024-03-13T07:36:45.959 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 0x56092676d620 2024-03-13T07:36:45.959 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: started 48 aios 2024-03-13T07:36:45.959 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092700de50 2024-03-13T07:36:45.959 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092648b5b0 2024-03-13T07:37:03.141 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:03:27.030 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:03.144 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:03:27.035 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:03.154 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:03:27.044 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:03.198 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:03:27.069 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:03.207 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:03:27.078 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:03.224 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:03:27.115 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:03.315 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:03:27.185 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:03.351 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:03:27.241 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:03.476 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:03:27.366 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56 api_pool: Running main() from gmock_main.cc 2024-03-13T07:37:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-03-13T07:37:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [----------] Global test environment set-up. 2024-03-13T07:37:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-03-13T07:37:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-03-13T07:37:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ OK ] LibRadosPools.PoolList (9001 ms) 2024-03-13T07:37:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-03-13T07:37:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (5698 ms) 2024-03-13T07:37:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-03-13T07:37:08.242 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (3693 ms) 2024-03-13T07:37:08.242 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-03-13T07:37:08.242 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (9196 ms) 2024-03-13T07:37:08.242 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-03-13T07:37:08.242 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (9820 ms) 2024-03-13T07:37:08.242 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-03-13T07:37:08.242 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (16431 ms) 2024-03-13T07:37:08.242 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-03-13T07:37:08.242 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (8784 ms) 2024-03-13T07:37:08.242 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-03-13T07:37:08.242 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (5684 ms) 2024-03-13T07:37:08.242 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-03-13T07:37:08.242 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (8149 ms) 2024-03-13T07:37:08.242 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [----------] 9 tests from LibRadosPools (76457 ms total) 2024-03-13T07:37:08.243 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: 2024-03-13T07:37:08.243 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [----------] Global test environment tear-down 2024-03-13T07:37:08.243 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (76457 ms total) 2024-03-13T07:37:08.243 INFO:tasks.workunit.client.0.smithi132.stdout: api_pool: [ PASSED ] 9 tests. 2024-03-13T07:37:22.716 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:37:22.500+0000 12a5a640 -1 osd.1 83 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4723.0:5 60.2 60:41780a1c:::tgt_object:head [call test_remote_reads.test_gather in=233b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56) 2024-03-13T07:37:23.497 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:03:47.387 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:23.522 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:03:47.412 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:23.549 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:03:47.420 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:23.559 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:03:47.449 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:23.563 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:03:47.453 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:23.672 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:03:47.542 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:23.735 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:03:47.625 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:23.811 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:03:47.681 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:23.823 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:03:47.714 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:24.233 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-03-13T07:37:24.217+0000 ea52640 -1 osd.3 86 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4774.0:2 73.15 73:ad551702:::test:head [create,setxattr key (5) in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) 2024-03-13T07:37:28.234 INFO:tasks.workunit.client.0.smithi132.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-03-13T07:37:28.234 INFO:tasks.workunit.client.0.smithi132.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-03-13T07:37:28.234 INFO:tasks.workunit.client.0.smithi132.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-03-13T07:37:28.235 INFO:tasks.workunit.client.0.smithi132.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-03-13T07:37:28.235 INFO:tasks.workunit.client.0.smithi132.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (96233 ms) 2024-03-13T07:37:28.235 INFO:tasks.workunit.client.0.smithi132.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (96233 ms total) 2024-03-13T07:37:28.235 INFO:tasks.workunit.client.0.smithi132.stdout: api_cls_remote_reads: 2024-03-13T07:37:28.235 INFO:tasks.workunit.client.0.smithi132.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-03-13T07:37:28.235 INFO:tasks.workunit.client.0.smithi132.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (96233 ms total) 2024-03-13T07:37:28.235 INFO:tasks.workunit.client.0.smithi132.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-03-13T07:37:31.826 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:31.754+0000 ea52640 -1 osd.7 94 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81.40e8aab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:31.930 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-03-13T07:37:31.842+0000 ea52640 -1 osd.3 92 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4774.0:2 73.15 73:ad551702:::test:head [create,setxattr key (5) in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) 2024-03-13T07:37:32.753 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:32.746+0000 ea52640 -1 osd.7 94 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81.40e8aab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:33.126 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-03-13T07:37:33.040+0000 ea52640 -1 osd.3 93 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4774.0:2 73.15 73:ad551702:::test:head [create,setxattr key (5) in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) 2024-03-13T07:37:33.784 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:33.776+0000 ea52640 -1 osd.7 94 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81.40e8aab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:34.234 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-03-13T07:37:34.224+0000 ea52640 -1 osd.3 94 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4774.0:2 73.15 73:ad551702:::test:head [create,setxattr key (5) in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) 2024-03-13T07:37:35.354 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-03-13T07:37:35.332+0000 ea52640 -1 osd.3 95 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4774.0:2 73.15 73:ad551702:::test:head [create,setxattr key (5) in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) 2024-03-13T07:37:35.899 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:35.776+0000 ea52640 -1 osd.7 97 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81.40e8aab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:36.563 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-03-13T07:37:36.472+0000 ea52640 -1 osd.3 97 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4774.0:2 73.15 73:ad551702:::test:head [create,setxattr key (5) in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) 2024-03-13T07:37:37.407 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:37.347+0000 ea52640 -1 osd.7 98 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81.40e8aab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:37.709 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-03-13T07:37:37.690+0000 ea52640 -1 osd.3 98 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4774.0:2 73.15 73:ad551702:::test:head [create,setxattr key (5) in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) 2024-03-13T07:37:38.447 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:38.424+0000 ea52640 -1 osd.7 99 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4715.0:2 75.0 75:0384ab8e:::oid:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59) 2024-03-13T07:37:40.154 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:40.150+0000 ea52640 -1 osd.7 100 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4715.0:2 75.0 75:0384ab8e:::oid:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59) 2024-03-13T07:37:40.849 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-03-13T07:37:40.792+0000 ea52640 -1 osd.3 102 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4774.0:2 73.15 73:ad551702:::test:head [create,setxattr key (5) in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) 2024-03-13T07:37:41.616 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:41.615+0000 ea52640 -1 osd.7 102 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81.40e8aab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:42.008 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-03-13T07:37:41.895+0000 ea52640 -1 osd.3 103 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4774.0:2 73.15 73:ad551702:::test:head [create,setxattr key (5) in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) 2024-03-13T07:37:42.792 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:42.678+0000 ea52640 -1 osd.7 103 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81.40e8aab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:43.708 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:04:07.598 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:43.715 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:04:07.605 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:43.719 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:04:07.589 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:43.722 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:04:07.593 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:43.770 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:04:07.660 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:43.773 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:04:07.663 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:43.903 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:04:07.773 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:43.912 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:04:07.802 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:43.971 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:04:07.862 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:37:44.041 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:44.023+0000 ea52640 -1 osd.7 104 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4715.0:2 75.0 75:0384ab8e:::oid:head [write 0~128 in=128b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59) 2024-03-13T07:37:44.927 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-03-13T07:37:44.928 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-03-13T07:37:44.928 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-03-13T07:37:44.928 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-03-13T07:37:44.928 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: seed 41267 2024-03-13T07:37:44.928 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-03-13T07:37:44.928 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (21573 ms) 2024-03-13T07:37:44.928 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-03-13T07:37:44.928 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (10435 ms) 2024-03-13T07:37:44.928 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-03-13T07:37:44.928 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (428 ms) 2024-03-13T07:37:44.928 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-03-13T07:37:44.928 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (1482 ms) 2024-03-13T07:37:44.929 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-03-13T07:37:44.929 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (11275 ms) 2024-03-13T07:37:44.929 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (45193 ms total) 2024-03-13T07:37:44.929 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: 2024-03-13T07:37:44.929 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-03-13T07:37:44.929 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-03-13T07:37:44.929 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (27311 ms) 2024-03-13T07:37:44.929 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-03-13T07:37:44.929 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (187 ms) 2024-03-13T07:37:44.929 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-03-13T07:37:44.929 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (1309 ms) 2024-03-13T07:37:44.929 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-03-13T07:37:44.929 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (20523 ms) 2024-03-13T07:37:44.930 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (49330 ms total) 2024-03-13T07:37:44.930 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: 2024-03-13T07:37:44.930 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-03-13T07:37:44.930 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (113070 ms total) 2024-03-13T07:37:44.930 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-03-13T07:37:45.442 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:45.415+0000 ea52640 -1 osd.7 106 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81.40e8aab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:47.031 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:46.934+0000 ea52640 -1 osd.7 107 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81.40e8aab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:47.981 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:47.975+0000 ea52640 -1 osd.7 108 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81.40e8aab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:48.972 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:48.966+0000 ea52640 -1 osd.7 109 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81.40e8aab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:49.947 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:49.943+0000 ea52640 -1 osd.7 109 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81.40e8aab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:51.282 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:51.276+0000 ea52640 -1 osd.7 110 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81.40e8aab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:52.289 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:52.284+0000 ea52640 -1 osd.7 110 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81.40e8aab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:53.129 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:37:52.976+0000 12a5a640 -1 osd.1 109 heartbeat_check: no reply from 172.21.15.195:6806 osd.6 since back 2024-03-13T07:37:52.189874+0000 front 2024-03-13T07:36:25.590195+0000 (oldest deadline 2024-03-13T07:37:52.180966+0000) 2024-03-13T07:37:53.323 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:53.318+0000 ea52640 -1 osd.7 110 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81.40e8aab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:54.617 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:37:54.430+0000 12a5a640 -1 osd.1 110 heartbeat_check: no reply from 172.21.15.195:6806 osd.6 since back 2024-03-13T07:37:53.740452+0000 front 2024-03-13T07:36:25.590195+0000 (oldest deadline 2024-03-13T07:37:52.180966+0000) 2024-03-13T07:37:55.121 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:55.001+0000 ea52640 -1 osd.7 111 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81.40e8aab5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:55.552 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:37:55.523+0000 12a5a640 -1 osd.1 111 heartbeat_check: no reply from 172.21.15.195:6806 osd.6 since back 2024-03-13T07:37:53.740452+0000 front 2024-03-13T07:36:25.590195+0000 (oldest deadline 2024-03-13T07:37:52.180966+0000) 2024-03-13T07:37:55.648 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:37:55.567+0000 12a5a640 -1 osd.1 111 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4723.0:5 60.2 60:41780a1c:::tgt_object:head [call test_remote_reads.test_gather in=233b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56) 2024-03-13T07:37:55.992 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:55.981+0000 ea52640 -1 osd.7 111 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81:ad551702:::test:head [create,setxattr key (5) in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:56.750 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:37:56.598+0000 12a5a640 -1 osd.1 111 heartbeat_check: no reply from 172.21.15.195:6806 osd.6 since back 2024-03-13T07:37:55.550640+0000 front 2024-03-13T07:36:25.590195+0000 (oldest deadline 2024-03-13T07:37:52.180966+0000) 2024-03-13T07:37:57.045 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:37:57.038+0000 ea52640 -1 osd.7 111 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4739.0:2 81.15 81:ad551702:::test:head [create,setxattr key (5) in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) 2024-03-13T07:37:57.124 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:37:56.935+0000 12a5a640 -1 osd.1 111 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4723.0:5 60.2 60:41780a1c:::tgt_object:head [call test_remote_reads.test_gather in=233b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56) 2024-03-13T07:37:57.981 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:37:57.960+0000 12a5a640 -1 osd.1 111 heartbeat_check: no reply from 172.21.15.195:6806 osd.6 since back 2024-03-13T07:37:55.550640+0000 front 2024-03-13T07:36:25.590195+0000 (oldest deadline 2024-03-13T07:37:52.180966+0000) 2024-03-13T07:37:58.049 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:37:58.013+0000 12a5a640 -1 osd.1 111 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4723.0:5 60.2 60:41780a1c:::tgt_object:head [call test_remote_reads.test_gather in=233b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56) 2024-03-13T07:38:08.983 INFO:tasks.workunit.client.0.smithi132.stdout: cls: Running main() from gmock_main.cc 2024-03-13T07:38:08.984 INFO:tasks.workunit.client.0.smithi132.stdout: cls: [==========] Running 2 tests from 1 test suite. 2024-03-13T07:38:08.984 INFO:tasks.workunit.client.0.smithi132.stdout: cls: [----------] Global test environment set-up. 2024-03-13T07:38:08.984 INFO:tasks.workunit.client.0.smithi132.stdout: cls: [----------] 2 tests from NeoRadosCls 2024-03-13T07:38:08.984 INFO:tasks.workunit.client.0.smithi132.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-03-13T07:38:08.984 INFO:tasks.workunit.client.0.smithi132.stdout: cls: [ OK ] NeoRadosCls.DNE (24138 ms) 2024-03-13T07:38:08.984 INFO:tasks.workunit.client.0.smithi132.stdout: cls: [ RUN ] NeoRadosCls.RemoteReads 2024-03-13T07:38:08.984 INFO:tasks.workunit.client.0.smithi132.stdout: cls: [ OK ] NeoRadosCls.RemoteReads (112826 ms) 2024-03-13T07:38:08.984 INFO:tasks.workunit.client.0.smithi132.stdout: cls: [----------] 2 tests from NeoRadosCls (136964 ms total) 2024-03-13T07:38:08.984 INFO:tasks.workunit.client.0.smithi132.stdout: cls: 2024-03-13T07:38:08.984 INFO:tasks.workunit.client.0.smithi132.stdout: cls: [----------] Global test environment tear-down 2024-03-13T07:38:08.984 INFO:tasks.workunit.client.0.smithi132.stdout: cls: [==========] 2 tests from 1 test suite ran. (136964 ms total) 2024-03-13T07:38:08.984 INFO:tasks.workunit.client.0.smithi132.stdout: cls: [ PASSED ] 2 tests. 2024-03-13T07:38:08.992 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-03-13T07:38:08.992 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-03-13T07:38:08.992 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-03-13T07:38:08.992 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-03-13T07:38:08.992 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-03-13T07:38:08.992 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-03-13T07:38:08.992 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-03-13T07:38:08.992 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (27362 ms) 2024-03-13T07:38:08.992 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-03-13T07:38:08.992 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (9806 ms) 2024-03-13T07:38:08.992 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-03-13T07:38:08.992 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (9329 ms) 2024-03-13T07:38:08.993 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-03-13T07:38:08.993 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (2150 ms) 2024-03-13T07:38:08.993 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-03-13T07:38:08.993 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (752 ms) 2024-03-13T07:38:08.993 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-03-13T07:38:08.993 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (4562 ms) 2024-03-13T07:38:08.993 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-03-13T07:38:08.993 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (6414 ms) 2024-03-13T07:38:08.993 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-03-13T07:38:08.993 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (11815 ms) 2024-03-13T07:38:08.993 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-03-13T07:38:08.993 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (10451 ms) 2024-03-13T07:38:08.993 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-03-13T07:38:08.993 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (7618 ms) 2024-03-13T07:38:08.993 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-03-13T07:38:08.994 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (30516 ms) 2024-03-13T07:38:08.994 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-03-13T07:38:08.994 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (128 ms) 2024-03-13T07:38:08.994 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-03-13T07:38:08.994 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (597 ms) 2024-03-13T07:38:08.994 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-03-13T07:38:08.994 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (2129 ms) 2024-03-13T07:38:08.994 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-03-13T07:38:08.994 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (547 ms) 2024-03-13T07:38:08.994 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-03-13T07:38:08.994 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (71 ms) 2024-03-13T07:38:08.994 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (124247 ms total) 2024-03-13T07:38:08.994 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: 2024-03-13T07:38:08.994 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-03-13T07:38:08.994 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (137022 ms total) 2024-03-13T07:38:08.995 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-03-13T07:38:12.165 INFO:tasks.workunit.client.0.smithi132.stdout: cmd: Running main() from gmock_main.cc 2024-03-13T07:38:12.165 INFO:tasks.workunit.client.0.smithi132.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-03-13T07:38:12.165 INFO:tasks.workunit.client.0.smithi132.stdout: cmd: [----------] Global test environment set-up. 2024-03-13T07:38:12.165 INFO:tasks.workunit.client.0.smithi132.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-03-13T07:38:12.166 INFO:tasks.workunit.client.0.smithi132.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-03-13T07:38:12.166 INFO:tasks.workunit.client.0.smithi132.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (5850 ms) 2024-03-13T07:38:12.166 INFO:tasks.workunit.client.0.smithi132.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-03-13T07:38:12.166 INFO:tasks.workunit.client.0.smithi132.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (13463 ms) 2024-03-13T07:38:12.166 INFO:tasks.workunit.client.0.smithi132.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-03-13T07:38:12.166 INFO:tasks.workunit.client.0.smithi132.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (120761 ms) 2024-03-13T07:38:12.166 INFO:tasks.workunit.client.0.smithi132.stdout: cmd: [----------] 3 tests from NeoRadosCmd (140074 ms total) 2024-03-13T07:38:12.166 INFO:tasks.workunit.client.0.smithi132.stdout: cmd: 2024-03-13T07:38:12.166 INFO:tasks.workunit.client.0.smithi132.stdout: cmd: [----------] Global test environment tear-down 2024-03-13T07:38:12.166 INFO:tasks.workunit.client.0.smithi132.stdout: cmd: [==========] 3 tests from 1 test suite ran. (140074 ms total) 2024-03-13T07:38:12.166 INFO:tasks.workunit.client.0.smithi132.stdout: cmd: [ PASSED ] 3 tests. 2024-03-13T07:38:13.137 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-03-13T07:38:13.137 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-03-13T07:38:13.138 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-03-13T07:38:13.138 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-03-13T07:38:13.138 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-03-13T07:38:13.138 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (968 ms) 2024-03-13T07:38:13.138 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-03-13T07:38:13.138 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (126 ms) 2024-03-13T07:38:13.138 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-03-13T07:38:13.138 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (140115 ms) 2024-03-13T07:38:13.138 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (141209 ms total) 2024-03-13T07:38:13.138 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd_pp: 2024-03-13T07:38:13.138 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-03-13T07:38:13.138 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (141209 ms total) 2024-03-13T07:38:13.138 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-03-13T07:38:16.190 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: Running main() from gmock_main.cc 2024-03-13T07:38:16.190 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-03-13T07:38:16.190 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [----------] Global test environment set-up. 2024-03-13T07:38:16.190 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-03-13T07:38:16.190 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-03-13T07:38:16.190 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ OK ] LibRadosStat.Stat (32913 ms) 2024-03-13T07:38:16.191 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-03-13T07:38:16.191 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (20453 ms) 2024-03-13T07:38:16.191 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-03-13T07:38:16.191 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ OK ] LibRadosStat.StatNS (31163 ms) 2024-03-13T07:38:16.191 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-03-13T07:38:16.191 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (4 ms) 2024-03-13T07:38:16.191 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-03-13T07:38:16.191 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (3510 ms) 2024-03-13T07:38:16.191 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [----------] 5 tests from LibRadosStat (88043 ms total) 2024-03-13T07:38:16.191 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: 2024-03-13T07:38:16.191 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-03-13T07:38:16.191 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-03-13T07:38:16.192 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (16629 ms) 2024-03-13T07:38:16.192 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-03-13T07:38:16.192 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (14402 ms) 2024-03-13T07:38:16.192 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-03-13T07:38:16.192 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (27 ms) 2024-03-13T07:38:16.192 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-03-13T07:38:16.192 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (758 ms) 2024-03-13T07:38:16.192 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (31816 ms total) 2024-03-13T07:38:16.192 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: 2024-03-13T07:38:16.192 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [----------] Global test environment tear-down 2024-03-13T07:38:16.192 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (144349 ms total) 2024-03-13T07:38:16.192 INFO:tasks.workunit.client.0.smithi132.stdout: api_stat: [ PASSED ] 9 tests. 2024-03-13T07:38:27.079 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-03-13T07:38:27.080 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [----------] Global test environment set-up. 2024-03-13T07:38:27.080 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [----------] 7 tests from LibRadosList 2024-03-13T07:38:27.080 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-03-13T07:38:27.080 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ OK ] LibRadosList.ListObjects (126643 ms) 2024-03-13T07:38:27.080 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-03-13T07:38:27.080 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (1231 ms) 2024-03-13T07:38:27.080 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-03-13T07:38:27.080 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: myset foo1,foo2,foo3 2024-03-13T07:38:27.080 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: foo1 2024-03-13T07:38:27.080 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: foo2 2024-03-13T07:38:27.081 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: foo3 2024-03-13T07:38:27.081 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: myset foo1,foo4,foo5 2024-03-13T07:38:27.081 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: foo4 2024-03-13T07:38:27.081 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: foo5 2024-03-13T07:38:27.081 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: foo1 2024-03-13T07:38:27.081 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: myset foo6,foo7 2024-03-13T07:38:27.081 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: foo7 2024-03-13T07:38:27.081 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: foo6 2024-03-13T07:38:27.081 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-03-13T07:38:27.081 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: ns1:foo4 2024-03-13T07:38:27.081 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: ns1:foo5 2024-03-13T07:38:27.081 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: ns2:foo7 2024-03-13T07:38:27.082 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: ns2:foo6 2024-03-13T07:38:27.082 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: ns1:foo1 2024-03-13T07:38:27.082 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: :foo1 2024-03-13T07:38:27.082 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: :foo2 2024-03-13T07:38:27.082 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: :foo3 2024-03-13T07:38:27.082 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (14369 ms) 2024-03-13T07:38:27.082 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-03-13T07:38:27.082 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 1 0 2024-03-13T07:38:27.082 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 10 0 2024-03-13T07:38:27.082 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 13 0 2024-03-13T07:38:27.082 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 7 0 2024-03-13T07:38:27.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 14 0 2024-03-13T07:38:27.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 0 0 2024-03-13T07:38:27.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 15 0 2024-03-13T07:38:27.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 11 0 2024-03-13T07:38:27.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 5 0 2024-03-13T07:38:27.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 8 0 2024-03-13T07:38:27.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 6 0 2024-03-13T07:38:27.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 3 0 2024-03-13T07:38:27.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 4 0 2024-03-13T07:38:27.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 12 0 2024-03-13T07:38:27.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 9 0 2024-03-13T07:38:27.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 2 0 2024-03-13T07:38:27.083 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-03-13T07:38:27.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (5148 ms) 2024-03-13T07:38:27.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-03-13T07:38:27.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: x cursor=MIN 2024-03-13T07:38:27.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > oid=1 cursor=5:02547ec2:::1:head 2024-03-13T07:38:27.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > oid=10 cursor=5:52ea6a34:::10:head 2024-03-13T07:38:27.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > oid=13 cursor=5:566253c9:::13:head 2024-03-13T07:38:27.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > oid=7 cursor=5:5c6b0b28:::7:head 2024-03-13T07:38:27.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > oid=14 cursor=5:62a1935d:::14:head 2024-03-13T07:38:27.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > oid=0 cursor=5:6cac518f:::0:head 2024-03-13T07:38:27.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > oid=15 cursor=5:863748b0:::15:head 2024-03-13T07:38:27.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > oid=11 cursor=5:89d3ae78:::11:head 2024-03-13T07:38:27.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > oid=5 cursor=5:b29083e3:::5:head 2024-03-13T07:38:27.084 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > oid=8 cursor=5:bd63b0f1:::8:head 2024-03-13T07:38:27.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > oid=6 cursor=5:c4fdafeb:::6:head 2024-03-13T07:38:27.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > oid=3 cursor=5:cfc208b3:::3:head 2024-03-13T07:38:27.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > oid=4 cursor=5:d83876eb:::4:head 2024-03-13T07:38:27.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > oid=12 cursor=5:de5d7c5f:::12:head 2024-03-13T07:38:27.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > oid=9 cursor=5:e960b815:::9:head 2024-03-13T07:38:27.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > oid=2 cursor=5:f905c69b:::2:head 2024-03-13T07:38:27.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: FIRST> seek to MIN oid=1 2024-03-13T07:38:27.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : oid=1 cursor=5:02547ec2:::1:head 2024-03-13T07:38:27.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:02547ec2:::1:head 2024-03-13T07:38:27.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:02547ec2:::1:head -> 1 2024-03-13T07:38:27.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : oid=10 cursor=5:52ea6a34:::10:head 2024-03-13T07:38:27.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:52ea6a34:::10:head 2024-03-13T07:38:27.085 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:52ea6a34:::10:head -> 10 2024-03-13T07:38:27.086 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : oid=13 cursor=5:566253c9:::13:head 2024-03-13T07:38:27.086 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:566253c9:::13:head 2024-03-13T07:38:27.086 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:566253c9:::13:head -> 13 2024-03-13T07:38:27.086 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : oid=7 cursor=5:5c6b0b28:::7:head 2024-03-13T07:38:27.086 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:5c6b0b28:::7:head 2024-03-13T07:38:27.086 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:5c6b0b28:::7:head -> 7 2024-03-13T07:38:29.141 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : oid=14 cursor=5:62a1935d:::14:head 2024-03-13T07:38:29.141 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:62a1935d:::14:head 2024-03-13T07:38:29.141 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:62a1935d:::14:head -> 14 2024-03-13T07:38:29.141 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : oid=0 cursor=5:6cac518f:::0:head 2024-03-13T07:38:29.141 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:6cac518f:::0:head 2024-03-13T07:38:29.141 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:6cac518f:::0:head -> 0 2024-03-13T07:38:29.142 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : oid=15 cursor=5:863748b0:::15:head 2024-03-13T07:38:29.142 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:863748b0:::15:head 2024-03-13T07:38:29.142 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:863748b0:::15:head -> 15 2024-03-13T07:38:29.142 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : oid=11 cursor=5:89d3ae78:::11:head 2024-03-13T07:38:29.142 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:89d3ae78:::11:head 2024-03-13T07:38:29.142 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:89d3ae78:::11:head -> 11 2024-03-13T07:38:29.142 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : oid=5 cursor=5:b29083e3:::5:head 2024-03-13T07:38:29.142 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:b29083e3:::5:head 2024-03-13T07:38:29.142 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:b29083e3:::5:head -> 5 2024-03-13T07:38:29.142 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : oid=8 cursor=5:bd63b0f1:::8:head 2024-03-13T07:38:29.142 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:bd63b0f1:::8:head 2024-03-13T07:38:29.142 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:bd63b0f1:::8:head -> 8 2024-03-13T07:38:29.142 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : oid=6 cursor=5:c4fdafeb:::6:head 2024-03-13T07:38:29.142 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:c4fdafeb:::6:head 2024-03-13T07:38:29.142 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:c4fdafeb:::6:head -> 6 2024-03-13T07:38:29.143 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : oid=3 cursor=5:cfc208b3:::3:head 2024-03-13T07:38:29.143 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:cfc208b3:::3:head 2024-03-13T07:38:29.143 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:cfc208b3:::3:head -> 3 2024-03-13T07:38:29.143 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : oid=4 cursor=5:d83876eb:::4:head 2024-03-13T07:38:29.143 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:d83876eb:::4:head 2024-03-13T07:38:29.143 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:d83876eb:::4:head -> 4 2024-03-13T07:38:29.143 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : oid=12 cursor=5:de5d7c5f:::12:head 2024-03-13T07:38:29.143 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:de5d7c5f:::12:head 2024-03-13T07:38:29.143 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:de5d7c5f:::12:head -> 12 2024-03-13T07:38:29.143 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : oid=9 cursor=5:e960b815:::9:head 2024-03-13T07:38:29.143 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:e960b815:::9:head 2024-03-13T07:38:29.143 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:e960b815:::9:head -> 9 2024-03-13T07:38:29.143 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : oid=2 cursor=5:f905c69b:::2:head 2024-03-13T07:38:29.143 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:f905c69b:::2:head 2024-03-13T07:38:29.144 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:f905c69b:::2:head -> 2 2024-03-13T07:38:29.144 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:52ea6a34:::10:head 2024-03-13T07:38:29.144 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : cursor()=5:52ea6a34:::10:head expected=5:52ea6a34:::10:head 2024-03-13T07:38:29.144 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:52ea6a34:::10:head -> 10 2024-03-13T07:38:29.144 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : entry=10 expected=10 2024-03-13T07:38:29.144 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:566253c9:::13:head 2024-03-13T07:38:29.144 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : cursor()=5:566253c9:::13:head expected=5:566253c9:::13:head 2024-03-13T07:38:29.144 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:566253c9:::13:head -> 13 2024-03-13T07:38:29.144 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : entry=13 expected=13 2024-03-13T07:38:29.144 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:6cac518f:::0:head 2024-03-13T07:38:29.144 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : cursor()=5:6cac518f:::0:head expected=5:6cac518f:::0:head 2024-03-13T07:38:29.144 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:6cac518f:::0:head -> 0 2024-03-13T07:38:29.144 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : entry=0 expected=0 2024-03-13T07:38:29.144 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:d83876eb:::4:head 2024-03-13T07:38:29.144 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : cursor()=5:d83876eb:::4:head expected=5:d83876eb:::4:head 2024-03-13T07:38:29.144 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:d83876eb:::4:head -> 4 2024-03-13T07:38:29.145 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : entry=4 expected=4 2024-03-13T07:38:29.145 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:cfc208b3:::3:head 2024-03-13T07:38:29.145 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : cursor()=5:cfc208b3:::3:head expected=5:cfc208b3:::3:head 2024-03-13T07:38:29.145 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:cfc208b3:::3:head -> 3 2024-03-13T07:38:29.145 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : entry=3 expected=3 2024-03-13T07:38:29.145 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:02547ec2:::1:head 2024-03-13T07:38:29.145 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : cursor()=5:02547ec2:::1:head expected=5:02547ec2:::1:head 2024-03-13T07:38:29.145 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:02547ec2:::1:head -> 1 2024-03-13T07:38:29.145 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : entry=1 expected=1 2024-03-13T07:38:29.145 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:c4fdafeb:::6:head 2024-03-13T07:38:29.145 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : cursor()=5:c4fdafeb:::6:head expected=5:c4fdafeb:::6:head 2024-03-13T07:38:29.145 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:c4fdafeb:::6:head -> 6 2024-03-13T07:38:29.145 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : entry=6 expected=6 2024-03-13T07:38:29.145 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:bd63b0f1:::8:head 2024-03-13T07:38:29.145 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : cursor()=5:bd63b0f1:::8:head expected=5:bd63b0f1:::8:head 2024-03-13T07:38:29.146 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:bd63b0f1:::8:head -> 8 2024-03-13T07:38:29.146 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : entry=8 expected=8 2024-03-13T07:38:29.146 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:b29083e3:::5:head 2024-03-13T07:38:35.054 INFO:tasks.workunit.client.0.smithi132.stdout: a list_parallel: process_1_[41719]: starting. 2024-03-13T07:38:35.054 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_1_[41719]: creating pool ceph_test_rados_list_parallel.smithi132-41663 2024-03-13T07:38:35.055 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_1_[41719]: created object 0... 2024-03-13T07:38:35.055 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_1_[41719]: created object 25... 2024-03-13T07:38:35.055 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_1_[41719]: created object 49... 2024-03-13T07:38:35.055 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_1_[41719]: finishing. 2024-03-13T07:38:35.055 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_1_[41719]: shutting down. 2024-03-13T07:38:35.055 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_2_[41720]: starting. 2024-03-13T07:38:35.055 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_2_[41720]: listing objects. 2024-03-13T07:38:35.055 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_2_[41720]: listed object 0... 2024-03-13T07:38:35.055 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_2_[41720]: listed object 25... 2024-03-13T07:38:35.055 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_2_[41720]: saw 50 objects 2024-03-13T07:38:35.055 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_2_[41720]: shutting down. 2024-03-13T07:38:35.055 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:35.055 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_3_[61063]: starting. 2024-03-13T07:38:35.055 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_3_[61063]: creating pool ceph_test_rados_list_parallel.smithi132-41663 2024-03-13T07:38:35.056 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_3_[61063]: created object 0... 2024-03-13T07:38:35.056 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_3_[61063]: created object 25... 2024-03-13T07:38:35.056 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_3_[61063]: created object 49... 2024-03-13T07:38:35.056 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_3_[61063]: finishing. 2024-03-13T07:38:35.056 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_3_[61063]: shutting down. 2024-03-13T07:38:35.056 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:35.056 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_4_[61064]: starting. 2024-03-13T07:38:35.056 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_4_[61064]: listing objects. 2024-03-13T07:38:35.056 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_4_[61064]: listed object 0... 2024-03-13T07:38:35.056 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_4_[61064]: listed object 25... 2024-03-13T07:38:35.056 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_4_[61064]: saw 42 objects 2024-03-13T07:38:35.056 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_4_[61064]: shutting down. 2024-03-13T07:38:35.056 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:35.056 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_5_[61065]: starting. 2024-03-13T07:38:35.057 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_5_[61065]: removed 25 objects... 2024-03-13T07:38:35.057 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_5_[61065]: removed half of the objects 2024-03-13T07:38:35.057 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_5_[61065]: removed 50 objects... 2024-03-13T07:38:35.057 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_5_[61065]: removed 50 objects 2024-03-13T07:38:35.057 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_5_[61065]: shutting down. 2024-03-13T07:38:35.057 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:35.057 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:35.057 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_6_[75189]: starting. 2024-03-13T07:38:35.057 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_6_[75189]: creating pool ceph_test_rados_list_parallel.smithi132-41663 2024-03-13T07:38:35.057 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_6_[75189]: created object 0... 2024-03-13T07:38:35.057 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_6_[75189]: created object 25... 2024-03-13T07:38:35.057 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_6_[75189]: created object 49... 2024-03-13T07:38:35.057 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_6_[75189]: finishing. 2024-03-13T07:38:35.057 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_6_[75189]: shutting down. 2024-03-13T07:38:35.057 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:35.058 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:35.058 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_7_[75190]: starting. 2024-03-13T07:38:35.058 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_7_[75190]: listing objects. 2024-03-13T07:38:35.058 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_7_[75190]: listed object 0... 2024-03-13T07:38:35.058 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_7_[75190]: listed object 25... 2024-03-13T07:38:35.058 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_7_[75190]: listed object 50... 2024-03-13T07:38:35.058 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_7_[75190]: saw 54 objects 2024-03-13T07:38:35.058 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_7_[75190]: shutting down. 2024-03-13T07:38:35.058 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:35.058 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:35.058 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_8_[75192]: starting. 2024-03-13T07:38:35.058 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_8_[75192]: added 25 objects... 2024-03-13T07:38:35.058 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_8_[75192]: added half of the objects 2024-03-13T07:38:35.058 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_8_[75192]: added 50 objects... 2024-03-13T07:38:35.059 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_8_[75192]: added 50 objects 2024-03-13T07:38:35.059 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_8_[75192]: shutting down. 2024-03-13T07:38:35.059 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:35.059 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.368 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.368 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_9_[82041]: starting. 2024-03-13T07:38:46.368 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_9_[82041]: creating pool ceph_test_rados_list_parallel.smithi132-41663 2024-03-13T07:38:46.368 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_9_[82041]: created object 0... 2024-03-13T07:38:46.368 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_9_[82041]: created object 25... 2024-03-13T07:38:46.368 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_9_[82041]: created object 49... 2024-03-13T07:38:46.368 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_9_[82041]: finishing. 2024-03-13T07:38:46.368 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_9_[82041]: shutting down. 2024-03-13T07:38:46.368 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.368 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.368 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.368 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_10_[82042]: starting. 2024-03-13T07:38:46.368 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_10_[82042]: listing objects. 2024-03-13T07:38:46.369 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_10_[82042]: listed object 0... 2024-03-13T07:38:46.369 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_10_[82042]: listed object 25... 2024-03-13T07:38:46.369 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_10_[82042]: listed object 50... 2024-03-13T07:38:46.369 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_10_[82042]: listed object 75... 2024-03-13T07:38:46.369 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_10_[82042]: saw 100 objects 2024-03-13T07:38:46.369 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_10_[82042]: shutting down. 2024-03-13T07:38:46.369 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.369 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.369 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.369 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_13_[82045]: starting. 2024-03-13T07:38:46.369 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_13_[82045]: removed 25 objects... 2024-03-13T07:38:46.369 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_13_[82045]: removed half of the objects 2024-03-13T07:38:46.369 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_13_[82045]: removed 50 objects... 2024-03-13T07:38:46.369 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_13_[82045]: removed 50 objects 2024-03-13T07:38:46.370 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_13_[82045]: shutting down. 2024-03-13T07:38:46.370 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.370 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.370 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.370 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_11_[82043]: starting. 2024-03-13T07:38:46.370 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_11_[82043]: added 25 objects... 2024-03-13T07:38:46.370 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_11_[82043]: added half of the objects 2024-03-13T07:38:46.370 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_11_[82043]: added 50 objects... 2024-03-13T07:38:46.370 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_11_[82043]: added 50 objects 2024-03-13T07:38:46.370 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_11_[82043]: shutting down. 2024-03-13T07:38:46.370 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.370 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.370 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.370 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_12_[82044]: starting. 2024-03-13T07:38:46.370 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_12_[82044]: added 25 objects... 2024-03-13T07:38:46.371 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_12_[82044]: added half of the objects 2024-03-13T07:38:46.371 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_12_[82044]: added 50 objects... 2024-03-13T07:38:46.371 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_12_[82044]: added 50 objects 2024-03-13T07:38:46.371 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_12_[82044]: shutting down. 2024-03-13T07:38:46.371 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.371 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.371 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.371 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.371 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_14_[89452]: starting. 2024-03-13T07:38:46.371 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_14_[89452]: creating pool ceph_test_rados_list_parallel.smithi132-41663 2024-03-13T07:38:46.371 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_14_[89452]: created object 0... 2024-03-13T07:38:46.371 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_14_[89452]: created object 25... 2024-03-13T07:38:46.371 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_14_[89452]: created object 49... 2024-03-13T07:38:46.371 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_14_[89452]: finishing. 2024-03-13T07:38:46.371 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_14_[89452]: shutting down. 2024-03-13T07:38:46.371 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.372 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.372 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.372 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:46.372 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_15_[89453]: starting. 2024-03-13T07:38:46.372 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_15_[89453]: listing objects. 2024-03-13T07:38:46.372 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_15_[89453]: listed object 0... 2024-03-13T07:38:56.958 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_15_[89453]: listed object 25... 2024-03-13T07:38:56.959 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_15_[89453]: listed object 50... 2024-03-13T07:38:56.959 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_15_[89453]: listed object 75... 2024-03-13T07:38:56.959 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_15_[89453]: listed object 100... 2024-03-13T07:38:56.959 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_15_[89453]: listed object 125... 2024-03-13T07:38:56.959 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_15_[89453]: saw 150 objects 2024-03-13T07:38:56.959 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_15_[89453]: shutting down. 2024-03-13T07:38:56.959 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:56.959 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:56.959 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:56.959 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:56.959 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_16_[89454]: starting. 2024-03-13T07:38:56.959 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_16_[89454]: added 25 objects... 2024-03-13T07:38:56.960 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_16_[89454]: added half of the objects 2024-03-13T07:38:56.960 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_16_[89454]: added 50 objects... 2024-03-13T07:38:56.960 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_16_[89454]: added 50 objects 2024-03-13T07:38:56.960 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: process_16_[89454]: shutting down. 2024-03-13T07:38:56.960 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:56.960 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:56.960 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:56.960 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:56.960 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******************************* 2024-03-13T07:38:56.960 INFO:tasks.workunit.client.0.smithi132.stdout: list_parallel: ******* SUCCESS ********** 2024-03-13T07:38:58.546 INFO:tasks.workunit.client.0.smithi132.stdout:09263db7b0 2024-03-13T07:38:58.546 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926570fe0 2024-03-13T07:38:58.546 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092676b0e0 2024-03-13T07:38:58.546 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092676bb10 2024-03-13T07:38:58.546 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092676c520 2024-03-13T07:38:58.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092676cfa0 2024-03-13T07:38:58.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092676d9f0 2024-03-13T07:38:58.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926761150 2024-03-13T07:38:58.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x5609266be4c0 2024-03-13T07:38:58.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092676ea30 2024-03-13T07:38:58.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092676f4d0 2024-03-13T07:38:58.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092676d780 2024-03-13T07:38:58.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926770360 2024-03-13T07:38:58.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092676f210 2024-03-13T07:38:58.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926770e70 2024-03-13T07:38:58.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926771900 2024-03-13T07:38:58.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x5609267700f0 2024-03-13T07:38:58.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926772820 2024-03-13T07:38:58.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926771690 2024-03-13T07:38:58.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926773700 2024-03-13T07:38:58.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926772580 2024-03-13T07:38:58.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926534180 2024-03-13T07:38:58.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x5609267734a0 2024-03-13T07:38:58.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926535080 2024-03-13T07:38:58.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926533ef0 2024-03-13T07:38:58.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926535bc0 2024-03-13T07:38:58.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926536630 2024-03-13T07:38:58.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926534e30 2024-03-13T07:38:58.548 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926537520 2024-03-13T07:38:58.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x5609265363f0 2024-03-13T07:38:58.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926538490 2024-03-13T07:38:58.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x5609265372c0 2024-03-13T07:38:58.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926539360 2024-03-13T07:38:58.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x5609265381a0 2024-03-13T07:38:58.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092653a280 2024-03-13T07:38:58.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x5609265390b0 2024-03-13T07:38:58.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092653adc0 2024-03-13T07:38:58.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092653b850 2024-03-13T07:38:58.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x560926539fe0 2024-03-13T07:38:58.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092653c740 2024-03-13T07:38:58.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092653b580 2024-03-13T07:38:58.549 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092653d280 2024-03-13T07:38:58.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092653dc80 2024-03-13T07:38:58.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092676ffe0 2024-03-13T07:38:58.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092676d890 2024-03-13T07:38:58.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: waiting 0x56092676d620 2024-03-13T07:38:58.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: done. 2024-03-13T07:38:58.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (59302 ms) 2024-03-13T07:38:58.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (59302 ms total) 2024-03-13T07:38:58.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: 2024-03-13T07:38:58.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-03-13T07:38:58.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-03-13T07:38:58.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-03-13T07:38:58.550 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-03-13T07:38:58.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ OK ] LibRadosMisc.Exec (18236 ms) 2024-03-13T07:38:58.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-03-13T07:38:58.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (22657 ms) 2024-03-13T07:38:58.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-03-13T07:38:58.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (3148 ms) 2024-03-13T07:38:58.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-03-13T07:38:58.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ OK ] LibRadosMisc.Applications (5621 ms) 2024-03-13T07:38:58.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-03-13T07:38:58.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-03-13T07:38:58.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-03-13T07:38:58.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-03-13T07:38:58.551 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-03-13T07:38:58.552 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (70860 ms) 2024-03-13T07:38:58.552 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [----------] 8 tests from LibRadosMisc (120522 ms total) 2024-03-13T07:38:58.552 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: 2024-03-13T07:38:58.552 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [----------] Global test environment tear-down 2024-03-13T07:38:58.552 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (186780 ms total) 2024-03-13T07:38:58.552 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc: [ PASSED ] 12 tests. 2024-03-13T07:38:58.556 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_1_[41743]: starting. 2024-03-13T07:38:58.556 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_1_[41743]: creating pool ceph_test_rados_open_pools_parallel.smithi132-41668 2024-03-13T07:38:58.556 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_1_[41743]: created object 0... 2024-03-13T07:38:58.556 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_1_[41743]: created object 25... 2024-03-13T07:38:58.556 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_1_[41743]: created object 49... 2024-03-13T07:38:58.556 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_1_[41743]: finishing. 2024-03-13T07:38:58.556 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_1_[41743]: shutting down. 2024-03-13T07:38:58.556 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_2_[41744]: starting. 2024-03-13T07:38:58.556 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_2_[41744]: rados_pool_create. 2024-03-13T07:38:58.557 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_2_[41744]: rados_ioctx_create. 2024-03-13T07:38:58.557 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_2_[41744]: shutting down. 2024-03-13T07:38:58.557 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: ******************************* 2024-03-13T07:38:58.557 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_3_[65333]: starting. 2024-03-13T07:38:58.557 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_3_[65333]: creating pool ceph_test_rados_open_pools_parallel.smithi132-41668 2024-03-13T07:38:58.557 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_3_[65333]: created object 0... 2024-03-13T07:38:58.557 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_3_[65333]: created object 25... 2024-03-13T07:38:58.557 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_3_[65333]: created object 49... 2024-03-13T07:38:58.557 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_3_[65333]: finishing. 2024-03-13T07:38:58.557 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_3_[65333]: shutting down. 2024-03-13T07:38:58.557 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: ******************************* 2024-03-13T07:38:58.558 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_4_[65334]: starting. 2024-03-13T07:38:58.558 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_4_[65334]: rados_pool_create. 2024-03-13T07:38:58.558 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_4_[65334]: rados_ioctx_create. 2024-03-13T07:38:58.558 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: process_4_[65334]: shutting down. 2024-03-13T07:38:58.558 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: ******************************* 2024-03-13T07:38:58.558 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: ******************************* 2024-03-13T07:38:58.558 INFO:tasks.workunit.client.0.smithi132.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-03-13T07:39:05.522 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:05:29.412 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:05.602 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:05:29.492 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:05.603 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:05:29.493 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:05.632 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:05:29.522 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:05.700 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:05:29.571 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:05.737 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:05:29.607 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:05.878 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:05:29.768 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:05.908 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:05:29.798 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:05.917 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:05:29.788 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:29.362 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-03-13T07:39:29.362 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-03-13T07:39:29.362 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-03-13T07:39:29.362 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-03-13T07:39:29.362 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-03-13T07:39:29.362 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: watch_notify_test_cb 2024-03-13T07:39:29.362 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (39754 ms) 2024-03-13T07:39:29.362 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-03-13T07:39:29.362 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-03-13T07:39:29.362 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94747519161328 err -107 2024-03-13T07:39:29.363 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (13216 ms) 2024-03-13T07:39:29.363 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-03-13T07:39:29.363 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-03-13T07:39:29.363 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94747519173136 err -107 2024-03-13T07:39:29.363 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (66412 ms) 2024-03-13T07:39:29.363 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-03-13T07:39:29.363 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: watch_notify2_test_cb from 4459 notify_id 476741369856 cookie 94747519189952 2024-03-13T07:39:29.363 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (1589 ms) 2024-03-13T07:39:29.363 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-03-13T07:39:29.363 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: watch_notify2_test_cb from 4459 notify_id 476741369856 cookie 94747519208624 2024-03-13T07:39:29.363 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (286 ms) 2024-03-13T07:39:29.363 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-03-13T07:39:29.363 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: watch_notify2_test_cb from 4459 notify_id 476741369857 cookie 94747519208560 2024-03-13T07:39:29.363 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (664 ms) 2024-03-13T07:39:29.364 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-03-13T07:39:29.364 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: watch_notify2_test_cb from 4459 notify_id 481036337152 cookie 94747519208560 2024-03-13T07:39:29.364 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: watch_notify2_test_cb from 4459 notify_id 481036337152 cookie 94747519240048 2024-03-13T07:39:29.364 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: watch_notify2_test_cb from 4459 notify_id 481036337153 cookie 94747519208560 2024-03-13T07:39:29.364 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: watch_notify2_test_cb from 4459 notify_id 481036337153 cookie 94747519240048 2024-03-13T07:39:29.364 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (2068 ms) 2024-03-13T07:39:29.364 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-03-13T07:39:29.364 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: watch_notify2_test_cb from 4459 notify_id 481036337153 cookie 94747519208560 2024-03-13T07:39:29.364 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: watch_notify2_test_cb from 4459 notify_id 481036337154 cookie 94747519208560 2024-03-13T07:39:29.364 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (5284 ms) 2024-03-13T07:39:29.364 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-03-13T07:39:29.364 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-03-13T07:39:29.364 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94747519208496 err -107 2024-03-13T07:39:29.364 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: watch_notify2_test_cb from 4459 notify_id 493921239041 cookie 94747519208496 2024-03-13T07:39:29.364 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (8151 ms) 2024-03-13T07:39:29.364 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-03-13T07:39:29.365 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-03-13T07:39:29.365 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94747519222768 err -107 2024-03-13T07:39:29.365 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (2231 ms) 2024-03-13T07:39:29.365 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (139655 ms total) 2024-03-13T07:39:29.365 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: 2024-03-13T07:39:29.365 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-03-13T07:39:29.365 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-03-13T07:39:29.365 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: watch_notify_test_cb 2024-03-13T07:39:29.365 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (24131 ms) 2024-03-13T07:39:29.365 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (24131 ms total) 2024-03-13T07:39:29.366 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: 2024-03-13T07:39:29.366 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-03-13T07:39:29.366 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (217491 ms total) 2024-03-13T07:39:29.366 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-03-13T07:39:31.071 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:05:54.961 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:31.140 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:05:55.010 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:31.141 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:05:55.012 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:31.144 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:05:55.035 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:31.160 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:05:55.051 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:31.164 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:05:55.055 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:31.266 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:05:55.137 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:31.281 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:05:55.171 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:31.452 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:05:55.342 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:45.055 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: Running main() from gmock_main.cc 2024-03-13T07:39:45.055 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-03-13T07:39:45.055 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [----------] Global test environment set-up. 2024-03-13T07:39:45.055 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-03-13T07:39:45.055 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: seed 40865 2024-03-13T07:39:45.055 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-03-13T07:39:45.056 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-03-13T07:39:45.056 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-03-13T07:39:45.056 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (57455 ms) 2024-03-13T07:39:45.056 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-03-13T07:39:45.056 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (8646 ms) 2024-03-13T07:39:45.056 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-03-13T07:39:45.056 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (23751 ms) 2024-03-13T07:39:45.056 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-03-13T07:39:45.056 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (30109 ms) 2024-03-13T07:39:45.056 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-03-13T07:39:45.056 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (59 ms) 2024-03-13T07:39:45.056 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-03-13T07:39:45.056 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (1632 ms) 2024-03-13T07:39:45.056 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-03-13T07:39:45.057 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (364 ms) 2024-03-13T07:39:45.057 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-03-13T07:39:45.057 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (80 ms) 2024-03-13T07:39:45.057 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-03-13T07:39:45.057 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (300 ms) 2024-03-13T07:39:45.057 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-03-13T07:39:45.057 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (43 ms) 2024-03-13T07:39:45.057 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-03-13T07:39:45.057 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (156 ms) 2024-03-13T07:39:45.057 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-03-13T07:39:45.057 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (263 ms) 2024-03-13T07:39:45.057 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-03-13T07:39:45.057 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (200 ms) 2024-03-13T07:39:45.057 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-03-13T07:39:45.057 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (1020 ms) 2024-03-13T07:39:45.058 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-03-13T07:39:45.058 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (1586 ms) 2024-03-13T07:39:45.058 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-03-13T07:39:45.058 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (1208 ms) 2024-03-13T07:39:45.058 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-03-13T07:39:45.058 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (268 ms) 2024-03-13T07:39:45.058 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-03-13T07:39:45.058 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (89 ms) 2024-03-13T07:39:45.058 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-03-13T07:39:45.058 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (142 ms) 2024-03-13T07:39:45.058 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (127371 ms total) 2024-03-13T07:39:45.058 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: 2024-03-13T07:39:45.058 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-03-13T07:39:45.058 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-03-13T07:39:45.059 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (10407 ms) 2024-03-13T07:39:45.059 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-03-13T07:39:45.059 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (28392 ms) 2024-03-13T07:39:45.059 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-03-13T07:39:45.059 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (2362 ms) 2024-03-13T07:39:45.059 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-03-13T07:39:45.059 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (4437 ms) 2024-03-13T07:39:45.059 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-03-13T07:39:47.279 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.Round api_lock: Running main() from gmock_main.cc 2024-03-13T07:39:47.280 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-03-13T07:39:47.280 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [----------] Global test environment set-up. 2024-03-13T07:39:47.280 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-03-13T07:39:47.280 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-03-13T07:39:47.280 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (44032 ms) 2024-03-13T07:39:47.280 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-03-13T07:39:47.280 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ OK ] LibRadosLock.LockShared (5788 ms) 2024-03-13T07:39:47.280 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-03-13T07:39:47.280 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (4647 ms) 2024-03-13T07:39:47.280 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-03-13T07:39:47.280 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1596 ms) 2024-03-13T07:39:47.280 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-03-13T07:39:47.280 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (64395 ms) 2024-03-13T07:39:47.281 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-03-13T07:39:47.281 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ OK ] LibRadosLock.Unlock (2253 ms) 2024-03-13T07:39:47.281 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-03-13T07:39:47.281 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (879 ms) 2024-03-13T07:39:47.281 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-03-13T07:39:47.281 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (87 ms) 2024-03-13T07:39:47.281 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [----------] 8 tests from LibRadosLock (123677 ms total) 2024-03-13T07:39:47.281 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: 2024-03-13T07:39:47.281 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-03-13T07:39:47.281 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-03-13T07:39:47.281 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (9297 ms) 2024-03-13T07:39:47.281 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-03-13T07:39:47.281 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (5326 ms) 2024-03-13T07:39:47.281 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-03-13T07:39:47.281 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (4903 ms) 2024-03-13T07:39:47.282 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-03-13T07:39:47.282 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (10380 ms) 2024-03-13T07:39:47.282 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-03-13T07:39:47.282 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (15352 ms) 2024-03-13T07:39:47.282 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-03-13T07:39:47.282 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (5747 ms) 2024-03-13T07:39:47.282 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-03-13T07:39:47.282 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (3152 ms) 2024-03-13T07:39:47.282 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-03-13T07:39:47.282 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (995 ms) 2024-03-13T07:39:47.282 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (55152 ms total) 2024-03-13T07:39:47.282 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: 2024-03-13T07:39:47.282 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [----------] Global test environment tear-down 2024-03-13T07:39:47.282 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (235522 ms total) 2024-03-13T07:39:47.282 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock: [ PASSED ] 16 tests. 2024-03-13T07:39:49.729 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-03-13T07:39:49.729 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-03-13T07:39:49.729 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-03-13T07:39:49.729 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-03-13T07:39:49.729 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: seed 40956 2024-03-13T07:39:49.729 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-03-13T07:39:49.729 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (37898 ms) 2024-03-13T07:39:49.729 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-03-13T07:39:49.729 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (7241 ms) 2024-03-13T07:39:49.729 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-03-13T07:39:49.730 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (8588 ms) 2024-03-13T07:39:49.730 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-03-13T07:39:49.730 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (46698 ms) 2024-03-13T07:39:49.730 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-03-13T07:39:49.730 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (7558 ms) 2024-03-13T07:39:49.730 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-03-13T07:39:49.730 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (11745 ms) 2024-03-13T07:39:49.730 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-03-13T07:39:49.730 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (173 ms) 2024-03-13T07:39:49.730 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-03-13T07:39:49.730 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (208 ms) 2024-03-13T07:39:49.731 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (120110 ms total) 2024-03-13T07:39:49.731 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: 2024-03-13T07:39:49.731 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-03-13T07:39:49.731 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-03-13T07:39:49.731 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (11991 ms) 2024-03-13T07:39:49.731 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-03-13T07:39:49.731 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (2470 ms) 2024-03-13T07:39:49.731 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-03-13T07:39:49.731 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (2706 ms) 2024-03-13T07:39:49.731 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-03-13T07:39:49.731 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (5498 ms) 2024-03-13T07:39:49.731 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-03-13T07:39:49.731 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (5496 ms) 2024-03-13T07:39:49.731 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-03-13T07:39:49.732 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (10743 ms) 2024-03-13T07:39:49.732 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-03-13T07:39:49.732 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (6343 ms) 2024-03-13T07:39:49.732 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-03-13T07:39:49.732 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (3727 ms) 2024-03-13T07:39:49.732 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (48975 ms total) 2024-03-13T07:39:49.732 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: 2024-03-13T07:39:49.732 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-03-13T07:39:49.732 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (237997 ms total) 2024-03-13T07:39:49.732 INFO:tasks.workunit.client.0.smithi132.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-03-13T07:39:51.026 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:06:14.916 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:51.091 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:06:14.981 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:51.159 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:06:15.050 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:51.163 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:06:15.034 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:51.167 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:06:15.058 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:51.177 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:06:15.048 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:51.236 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:06:15.127 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:51.284 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:06:15.155 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:39:51.291 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:06:15.181 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:06.297 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: Running main() from gmock_main.cc 2024-03-13T07:40:06.297 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-03-13T07:40:06.297 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [----------] Global test environment set-up. 2024-03-13T07:40:06.297 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-03-13T07:40:06.297 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-03-13T07:40:06.297 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (39081 ms) 2024-03-13T07:40:06.297 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-03-13T07:40:06.297 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-03-13T07:40:06.297 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-03-13T07:40:06.297 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: no timeout :/ 2024-03-13T07:40:06.298 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: no timeout :/ 2024-03-13T07:40:06.298 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: no timeout :/ 2024-03-13T07:40:06.298 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: no timeout :/ 2024-03-13T07:40:06.298 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: no timeout :/ 2024-03-13T07:40:06.298 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (4013 ms) 2024-03-13T07:40:06.298 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-03-13T07:40:06.298 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (5460 ms) 2024-03-13T07:40:06.298 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-03-13T07:40:06.298 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIo.Checksum (11843 ms) 2024-03-13T07:40:06.298 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-03-13T07:40:06.298 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (11211 ms) 2024-03-13T07:40:06.298 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-03-13T07:40:06.298 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (12458 ms) 2024-03-13T07:40:06.298 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-03-13T07:40:06.299 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (7620 ms) 2024-03-13T07:40:06.299 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-03-13T07:40:06.299 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (28524 ms) 2024-03-13T07:40:06.299 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-03-13T07:40:06.299 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIo.TruncTest (2222 ms) 2024-03-13T07:40:06.299 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-03-13T07:40:06.299 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (543 ms) 2024-03-13T07:40:06.299 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-03-13T07:40:06.299 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (156 ms) 2024-03-13T07:40:06.299 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-03-13T07:40:06.299 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIo.RmXattr (3872 ms) 2024-03-13T07:40:06.299 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-03-13T07:40:06.299 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIo.XattrIter (406 ms) 2024-03-13T07:40:06.299 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [----------] 14 tests from LibRadosIo (127409 ms total) 2024-03-13T07:40:06.299 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: 2024-03-13T07:40:06.300 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-03-13T07:40:06.300 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-03-13T07:40:06.300 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (11132 ms) 2024-03-13T07:40:06.300 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-03-13T07:40:06.300 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (5157 ms) 2024-03-13T07:40:06.300 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-03-13T07:40:06.300 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (5588 ms) 2024-03-13T07:40:06.300 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-03-13T07:40:06.300 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (6347 ms) 2024-03-13T07:40:06.300 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-03-13T07:40:06.300 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (12366 ms) 2024-03-13T07:40:06.300 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-03-13T07:40:06.300 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (3483 ms) 2024-03-13T07:40:06.300 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-03-13T07:40:06.300 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (4505 ms) 2024-03-13T07:40:06.300 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-03-13T07:40:06.301 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (4192 ms) 2024-03-13T07:40:06.301 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-03-13T07:40:06.301 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (9774 ms) 2024-03-13T07:40:06.301 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-03-13T07:40:06.301 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (4128 ms) 2024-03-13T07:40:06.301 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [----------] 10 tests from LibRadosIoEC (66672 ms total) 2024-03-13T07:40:06.301 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: 2024-03-13T07:40:06.301 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [----------] Global test environment tear-down 2024-03-13T07:40:06.301 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [==========] 24 tests from 2 test suites ran. (254616 ms total) 2024-03-13T07:40:06.302 INFO:tasks.workunit.client.0.smithi132.stdout: api_io: [ PASSED ] 24 tests. 2024-03-13T07:40:11.573 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:06:35.463 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:11.581 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:06:35.470 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:11.586 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:06:35.457 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:11.602 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:06:35.472 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:11.660 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:06:35.551 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:11.664 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:06:35.554 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:11.694 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:06:35.564 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:11.714 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:06:35.604 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:11.782 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:06:35.672 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:16.999 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: Running main() from gmock_main.cc 2024-03-13T07:40:16.999 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-03-13T07:40:16.999 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-03-13T07:40:16.999 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-03-13T07:40:16.999 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: seed 41222 2024-03-13T07:40:16.999 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-03-13T07:40:17.000 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (26579 ms) 2024-03-13T07:40:17.000 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-03-13T07:40:17.000 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (12165 ms) 2024-03-13T07:40:17.000 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-03-13T07:40:17.000 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (11437 ms) 2024-03-13T07:40:17.000 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-03-13T07:40:17.000 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (6837 ms) 2024-03-13T07:40:17.000 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-03-13T07:40:17.000 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (38184 ms) 2024-03-13T07:40:17.000 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (95203 ms total) 2024-03-13T07:40:17.000 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: 2024-03-13T07:40:17.000 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-03-13T07:40:17.000 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-03-13T07:40:17.000 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (14020 ms) 2024-03-13T07:40:17.000 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-03-13T07:40:17.001 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (31994 ms) 2024-03-13T07:40:17.001 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-03-13T07:40:17.001 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (54494 ms) 2024-03-13T07:40:17.001 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-03-13T07:40:17.001 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (8116 ms) 2024-03-13T07:40:17.001 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-03-13T07:40:17.001 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (14038 ms) 2024-03-13T07:40:17.001 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-13T07:40:17.001 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2007-gb7c579c9/rpm/el9/BUILD/ceph-19.0.0-2007-gb7c579c9/src/test/librados/snapshots_cxx.cc:463: Skipped 2024-03-13T07:40:17.001 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: 2024-03-13T07:40:17.001 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-03-13T07:40:17.001 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-03-13T07:40:17.001 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi132-41222-7 2024-03-13T07:40:17.001 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: waiting for snaps to purge 2024-03-13T07:40:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (20394 ms) 2024-03-13T07:40:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (143056 ms total) 2024-03-13T07:40:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: 2024-03-13T07:40:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-03-13T07:40:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-03-13T07:40:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (8 ms) 2024-03-13T07:40:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-03-13T07:40:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5083 ms) 2024-03-13T07:40:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5091 ms total) 2024-03-13T07:40:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: 2024-03-13T07:40:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-03-13T07:40:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-03-13T07:40:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (7454 ms) 2024-03-13T07:40:17.002 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-03-13T07:40:19.107 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] io: Running main() from gmock_main.cc 2024-03-13T07:40:19.107 INFO:tasks.workunit.client.0.smithi132.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-03-13T07:40:19.107 INFO:tasks.workunit.client.0.smithi132.stdout: io: [----------] Global test environment set-up. 2024-03-13T07:40:19.107 INFO:tasks.workunit.client.0.smithi132.stdout: io: [----------] 14 tests from NeoRadosIo 2024-03-13T07:40:19.107 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-03-13T07:40:19.107 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ OK ] NeoRadosIo.Limits (27277 ms) 2024-03-13T07:40:19.108 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-03-13T07:40:19.108 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (18839 ms) 2024-03-13T07:40:19.108 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-03-13T07:40:19.108 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ OK ] NeoRadosIo.ReadOp (22029 ms) 2024-03-13T07:40:19.108 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-03-13T07:40:19.108 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ OK ] NeoRadosIo.SparseRead (37237 ms) 2024-03-13T07:40:19.108 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-03-13T07:40:19.108 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ OK ] NeoRadosIo.RoundTrip (29636 ms) 2024-03-13T07:40:19.108 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-03-13T07:40:19.108 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (48578 ms) 2024-03-13T07:40:19.108 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-03-13T07:40:19.108 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (24468 ms) 2024-03-13T07:40:19.109 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-03-13T07:40:19.109 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (4469 ms) 2024-03-13T07:40:19.109 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-03-13T07:40:19.109 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (14559 ms) 2024-03-13T07:40:19.109 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-03-13T07:40:19.109 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ OK ] NeoRadosIo.Trunc (9276 ms) 2024-03-13T07:40:19.109 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-03-13T07:40:19.109 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ OK ] NeoRadosIo.Remove (5720 ms) 2024-03-13T07:40:19.109 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-03-13T07:40:19.109 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (10293 ms) 2024-03-13T07:40:19.109 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-03-13T07:40:19.109 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ OK ] NeoRadosIo.RmXattr (9288 ms) 2024-03-13T07:40:19.109 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-03-13T07:40:19.109 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ OK ] NeoRadosIo.GetXattrs (5339 ms) 2024-03-13T07:40:19.110 INFO:tasks.workunit.client.0.smithi132.stdout: io: [----------] 14 tests from NeoRadosIo (267008 ms total) 2024-03-13T07:40:19.110 INFO:tasks.workunit.client.0.smithi132.stdout: io: 2024-03-13T07:40:19.110 INFO:tasks.workunit.client.0.smithi132.stdout: io: [----------] Global test environment tear-down 2024-03-13T07:40:19.110 INFO:tasks.workunit.client.0.smithi132.stdout: io: [==========] 14 tests from 1 test suite ran. (267008 ms total) 2024-03-13T07:40:19.110 INFO:tasks.workunit.client.0.smithi132.stdout: io: [ PASSED ] 14 tests. 2024-03-13T07:40:31.736 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:06:55.627 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:31.746 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:06:55.617 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:31.750 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:06:55.640 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:31.752 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:06:55.623 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:31.753 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:06:55.643 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:31.756 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:06:55.646 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:31.915 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:06:55.785 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:31.919 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:06:55.809 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:31.939 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:06:55.830 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:33.015 INFO:tasks.workunit.client.0.smithi132.stdout:TripPP2 (930 ms) 2024-03-13T07:40:33.015 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-03-13T07:40:33.015 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (4761 ms) 2024-03-13T07:40:33.016 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-03-13T07:40:33.016 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (7019 ms) 2024-03-13T07:40:33.016 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-03-13T07:40:33.016 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (2521 ms) 2024-03-13T07:40:33.016 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-03-13T07:40:33.016 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (3685 ms) 2024-03-13T07:40:33.016 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-03-13T07:40:33.016 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (2762 ms) 2024-03-13T07:40:33.016 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-03-13T07:40:33.016 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (2385 ms) 2024-03-13T07:40:33.016 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-03-13T07:40:33.016 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (2181 ms) 2024-03-13T07:40:33.016 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-03-13T07:40:33.016 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (7282 ms) 2024-03-13T07:40:33.017 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-03-13T07:40:33.017 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (6152 ms) 2024-03-13T07:40:33.017 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-03-13T07:40:33.017 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (3875 ms) 2024-03-13T07:40:33.017 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-03-13T07:40:33.017 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (899 ms) 2024-03-13T07:40:33.017 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-03-13T07:40:33.017 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (1273 ms) 2024-03-13T07:40:33.017 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (91323 ms total) 2024-03-13T07:40:33.017 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: 2024-03-13T07:40:33.017 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [----------] Global test environment tear-down 2024-03-13T07:40:33.017 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (281334 ms total) 2024-03-13T07:40:33.017 INFO:tasks.workunit.client.0.smithi132.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-03-13T07:40:38.422 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_1_[41709]: starting. 2024-03-13T07:40:38.422 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_1_[41709]: creating pool ceph_test_rados_delete_pools_parallel.smithi132-41674 2024-03-13T07:40:38.423 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_1_[41709]: created object 0... 2024-03-13T07:40:38.423 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_1_[41709]: created object 25... 2024-03-13T07:40:38.423 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_1_[41709]: created object 49... 2024-03-13T07:40:38.423 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_1_[41709]: finishing. 2024-03-13T07:40:38.423 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_1_[41709]: shutting down. 2024-03-13T07:40:38.423 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_2_[41710]: starting. 2024-03-13T07:40:38.423 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_2_[41710]: deleting pool ceph_test_rados_delete_pools_parallel.smithi132-41674 2024-03-13T07:40:38.423 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_2_[41710]: shutting down. 2024-03-13T07:40:38.423 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: ******************************* 2024-03-13T07:40:38.423 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_3_[67762]: starting. 2024-03-13T07:40:38.423 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_3_[67762]: creating pool ceph_test_rados_delete_pools_parallel.smithi132-41674 2024-03-13T07:40:38.423 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_3_[67762]: created object 0... 2024-03-13T07:40:38.423 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_3_[67762]: created object 25... 2024-03-13T07:40:38.423 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_3_[67762]: created object 49... 2024-03-13T07:40:38.423 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_3_[67762]: finishing. 2024-03-13T07:40:38.424 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_3_[67762]: shutting down. 2024-03-13T07:40:38.424 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: ******************************* 2024-03-13T07:40:38.424 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_5_[67764]: starting. 2024-03-13T07:40:38.424 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_5_[67764]: listing objects. 2024-03-13T07:40:38.424 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_5_[67764]: listed object 0... 2024-03-13T07:40:38.424 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_5_[67764]: listed object 25... 2024-03-13T07:40:38.424 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_5_[67764]: saw 50 objects 2024-03-13T07:40:38.424 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_5_[67764]: shutting down. 2024-03-13T07:40:38.424 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: ******************************* 2024-03-13T07:40:38.424 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_4_[67763]: starting. 2024-03-13T07:40:38.424 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_4_[67763]: deleting pool ceph_test_rados_delete_pools_parallel.smithi132-41674 2024-03-13T07:40:38.424 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: process_4_[67763]: shutting down. 2024-03-13T07:40:38.424 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: ******************************* 2024-03-13T07:40:38.424 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: ******************************* 2024-03-13T07:40:38.424 INFO:tasks.workunit.client.0.smithi132.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-03-13T07:40:46.782 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: Running main() from gmock_main.cc 2024-03-13T07:40:46.782 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-03-13T07:40:46.782 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [----------] Global test environment set-up. 2024-03-13T07:40:46.782 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-03-13T07:40:46.782 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-03-13T07:40:46.782 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (40895 ms) 2024-03-13T07:40:46.782 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-03-13T07:40:46.783 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (11736 ms) 2024-03-13T07:40:46.783 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-03-13T07:40:46.783 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (23833 ms) 2024-03-13T07:40:46.783 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-03-13T07:40:46.783 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (7931 ms) 2024-03-13T07:40:46.783 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (84396 ms total) 2024-03-13T07:40:46.783 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: 2024-03-13T07:40:46.783 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-03-13T07:40:46.783 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-03-13T07:40:46.783 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (54449 ms) 2024-03-13T07:40:46.783 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-03-13T07:40:46.783 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (51751 ms) 2024-03-13T07:40:46.783 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-03-13T07:40:46.783 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (26451 ms) 2024-03-13T07:40:46.783 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (132651 ms total) 2024-03-13T07:40:46.784 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: 2024-03-13T07:40:46.784 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-03-13T07:40:46.784 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-03-13T07:40:46.784 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (8254 ms) 2024-03-13T07:40:46.784 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-03-13T07:40:46.784 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (5637 ms) 2024-03-13T07:40:46.784 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-03-13T07:40:46.784 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (12167 ms) 2024-03-13T07:40:46.784 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-03-13T07:40:46.784 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (6572 ms) 2024-03-13T07:40:46.784 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (32630 ms total) 2024-03-13T07:40:46.784 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: 2024-03-13T07:40:46.784 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-03-13T07:40:46.784 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-03-13T07:40:46.784 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (13733 ms) 2024-03-13T07:40:46.785 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-03-13T07:40:46.785 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (11313 ms) 2024-03-13T07:40:46.785 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (25046 ms total) 2024-03-13T07:40:46.785 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: 2024-03-13T07:40:46.785 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [----------] Global test environment tear-down 2024-03-13T07:40:46.785 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (294959 ms total) 2024-03-13T07:40:46.785 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-03-13T07:40:51.836 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: Running main() from gmock_main.cc 2024-03-13T07:40:51.836 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-03-13T07:40:51.836 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [----------] Global test environment set-up. 2024-03-13T07:40:51.836 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-03-13T07:40:51.836 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-03-13T07:40:51.836 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (47389 ms) 2024-03-13T07:40:51.837 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-03-13T07:40:51.837 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (84819 ms) 2024-03-13T07:40:51.837 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-03-13T07:40:51.837 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (36905 ms) 2024-03-13T07:40:51.837 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-03-13T07:40:51.837 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (46920 ms) 2024-03-13T07:40:51.837 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-03-13T07:40:51.837 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (8847 ms) 2024-03-13T07:40:51.837 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-03-13T07:40:51.837 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (8741 ms) 2024-03-13T07:40:51.837 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-03-13T07:40:51.837 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (7355 ms) 2024-03-13T07:40:51.837 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-03-13T07:40:51.837 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (6268 ms) 2024-03-13T07:40:51.838 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-03-13T07:40:51.838 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (6868 ms) 2024-03-13T07:40:51.838 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-03-13T07:40:51.838 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (7463 ms) 2024-03-13T07:40:51.838 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-03-13T07:40:51.838 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (16171 ms) 2024-03-13T07:40:51.838 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-03-13T07:40:51.838 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (9680 ms) 2024-03-13T07:40:51.838 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-03-13T07:40:51.838 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (5412 ms) 2024-03-13T07:40:51.838 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-03-13T07:40:51.838 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (6819 ms) 2024-03-13T07:40:51.838 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (299657 ms total) 2024-03-13T07:40:51.838 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: 2024-03-13T07:40:51.839 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [----------] Global test environment tear-down 2024-03-13T07:40:51.839 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (299657 ms total) 2024-03-13T07:40:51.839 INFO:tasks.workunit.client.0.smithi132.stdout: read_operations: [ PASSED ] 14 tests. 2024-03-13T07:40:51.839 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:07:15.728 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:51.909 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:07:15.799 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:51.920 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:07:15.790 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:51.925 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:07:15.796 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:51.947 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:07:15.837 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:51.981 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:07:15.872 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:52.002 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:07:15.892 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:52.015 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:07:15.886 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:40:52.066 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:07:15.956 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:00.970 INFO:tasks.workunit.client.0.smithi132.stdout:LibRadosSnapshotsECPP.SnapRemovePP (3252 ms) 2024-03-13T07:41:00.970 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-03-13T07:41:00.970 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (8817 ms) 2024-03-13T07:41:00.970 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-03-13T07:41:00.970 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2910 ms) 2024-03-13T07:41:00.970 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (22433 ms total) 2024-03-13T07:41:00.970 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: 2024-03-13T07:41:00.970 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-03-13T07:41:00.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-03-13T07:41:00.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (8533 ms) 2024-03-13T07:41:00.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-03-13T07:41:00.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (8267 ms) 2024-03-13T07:41:00.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-03-13T07:41:00.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (6127 ms) 2024-03-13T07:41:00.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (22927 ms total) 2024-03-13T07:41:00.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: 2024-03-13T07:41:00.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-03-13T07:41:00.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (309135 ms total) 2024-03-13T07:41:00.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-03-13T07:41:00.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-03-13T07:41:00.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-13T07:41:02.604 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: Running main() from gmock_main.cc 2024-03-13T07:41:02.604 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-03-13T07:41:02.605 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [----------] Global test environment set-up. 2024-03-13T07:41:02.605 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-03-13T07:41:02.605 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-03-13T07:41:02.605 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (37008 ms) 2024-03-13T07:41:02.605 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-03-13T07:41:02.605 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (21000 ms) 2024-03-13T07:41:02.605 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-03-13T07:41:02.605 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (86268 ms) 2024-03-13T07:41:02.605 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-03-13T07:41:02.605 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (45924 ms) 2024-03-13T07:41:02.605 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-03-13T07:41:02.605 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (13387 ms) 2024-03-13T07:41:02.605 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (203587 ms total) 2024-03-13T07:41:02.605 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: 2024-03-13T07:41:02.605 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-03-13T07:41:02.606 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-03-13T07:41:02.606 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (14632 ms) 2024-03-13T07:41:02.606 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-03-13T07:41:02.606 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (18064 ms) 2024-03-13T07:41:02.606 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-03-13T07:41:02.606 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (32760 ms) 2024-03-13T07:41:02.606 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-03-13T07:41:02.606 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (9919 ms) 2024-03-13T07:41:02.606 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-03-13T07:41:02.606 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (9699 ms) 2024-03-13T07:41:02.606 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-03-13T07:41:02.606 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi132-42029-11. 2024-03-13T07:41:02.606 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: Waiting for snaps to purge. 2024-03-13T07:41:02.607 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (21762 ms) 2024-03-13T07:41:02.607 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (106836 ms total) 2024-03-13T07:41:02.607 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: 2024-03-13T07:41:02.607 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [----------] Global test environment tear-down 2024-03-13T07:41:02.607 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (310423 ms total) 2024-03-13T07:41:02.607 INFO:tasks.workunit.client.0.smithi132.stdout: snapshots: [ PASSED ] 11 tests. 2024-03-13T07:41:11.907 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:07:35.798 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:11.970 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:07:35.860 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:11.975 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:07:35.865 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:11.978 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:07:35.848 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:11.981 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:07:35.851 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:11.991 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:07:35.881 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:12.063 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:07:35.934 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:12.077 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:07:35.967 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:12.131 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:07:36.022 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:25.375 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: Running main() from gmock_main.cc 2024-03-13T07:41:25.375 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-03-13T07:41:25.375 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [----------] Global test environment set-up. 2024-03-13T07:41:25.375 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-03-13T07:41:25.375 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-03-13T07:41:25.375 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.TooBig (4919 ms) 2024-03-13T07:41:25.375 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-03-13T07:41:25.375 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (84211 ms) 2024-03-13T07:41:25.375 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-03-13T07:41:25.376 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (34563 ms) 2024-03-13T07:41:25.376 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-03-13T07:41:25.376 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (46893 ms) 2024-03-13T07:41:25.376 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-03-13T07:41:25.376 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (27569 ms) 2024-03-13T07:41:25.376 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-03-13T07:41:25.376 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (8232 ms) 2024-03-13T07:41:25.376 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-03-13T07:41:25.376 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (9095 ms) 2024-03-13T07:41:25.376 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-03-13T07:41:25.376 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (12031 ms) 2024-03-13T07:41:25.376 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-03-13T07:41:25.376 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (14994 ms) 2024-03-13T07:41:25.376 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-03-13T07:41:25.376 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (14241 ms) 2024-03-13T07:41:25.377 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-03-13T07:41:25.377 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (6419 ms) 2024-03-13T07:41:25.377 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-03-13T07:41:25.377 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (7784 ms) 2024-03-13T07:41:25.377 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-03-13T07:41:25.377 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (6115 ms) 2024-03-13T07:41:25.377 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-03-13T07:41:25.377 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (5402 ms) 2024-03-13T07:41:25.377 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-03-13T07:41:25.377 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.Flush (4389 ms) 2024-03-13T07:41:25.377 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-03-13T07:41:25.377 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (6495 ms) 2024-03-13T07:41:25.377 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-03-13T07:41:25.377 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (4950 ms) 2024-03-13T07:41:25.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-03-13T07:41:25.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (4047 ms) 2024-03-13T07:41:25.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-03-13T07:41:25.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (4841 ms) 2024-03-13T07:41:25.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-03-13T07:41:25.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3744 ms) 2024-03-13T07:41:25.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-03-13T07:41:25.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (3323 ms) 2024-03-13T07:41:25.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-03-13T07:41:25.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3354 ms) 2024-03-13T07:41:25.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-03-13T07:41:25.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3924 ms) 2024-03-13T07:41:25.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-03-13T07:41:25.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (2985 ms) 2024-03-13T07:41:25.378 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-03-13T07:41:25.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3215 ms) 2024-03-13T07:41:25.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-03-13T07:41:25.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (5974 ms) 2024-03-13T07:41:25.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [----------] 26 tests from LibRadosAio (333710 ms total) 2024-03-13T07:41:25.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: 2024-03-13T07:41:25.379 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-03-13T07:41:36.982 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:08:00.872 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:36.989 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:08:00.879 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:37.045 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:08:00.916 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:37.048 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:08:00.939 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:37.050 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:08:00.921 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:37.058 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:08:00.949 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:37.071 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:08:00.961 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:37.110 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:08:00.981 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:37.132 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:08:01.021 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:52.511 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:08:16.401 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:52.516 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:08:16.406 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:52.522 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:08:16.393 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:52.524 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:08:16.395 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:52.531 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:08:16.421 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:52.532 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:08:16.423 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:52.636 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:08:16.507 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:52.655 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:08:16.545 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:52.664 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:08:16.555 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:57.005 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:08:20.895 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:57.077 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:08:20.967 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:57.100 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:08:20.990 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:57.141 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:08:21.011 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:57.142 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:08:21.013 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:57.148 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:08:21.038 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:57.149 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:08:21.040 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:57.168 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:08:21.039 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:41:57.170 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:08:21.060 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:02.258 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAioEC.Simplpi_list: : cursor()=5:b29083e3:::5:head expected=5:b29083e3:::5:head 2024-03-13T07:42:02.258 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:b29083e3:::5:head -> 5 2024-03-13T07:42:02.258 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : entry=5 expected=5 2024-03-13T07:42:02.258 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:89d3ae78:::11:head 2024-03-13T07:42:02.258 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : cursor()=5:89d3ae78:::11:head expected=5:89d3ae78:::11:head 2024-03-13T07:42:02.259 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:89d3ae78:::11:head -> 11 2024-03-13T07:42:02.259 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : entry=11 expected=11 2024-03-13T07:42:02.259 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:863748b0:::15:head 2024-03-13T07:42:02.259 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : cursor()=5:863748b0:::15:head expected=5:863748b0:::15:head 2024-03-13T07:42:02.259 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:863748b0:::15:head -> 15 2024-03-13T07:42:02.259 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : entry=15 expected=15 2024-03-13T07:42:02.259 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:62a1935d:::14:head 2024-03-13T07:42:02.259 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : cursor()=5:62a1935d:::14:head expected=5:62a1935d:::14:head 2024-03-13T07:42:02.259 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:62a1935d:::14:head -> 14 2024-03-13T07:42:02.259 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : entry=14 expected=14 2024-03-13T07:42:02.259 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:f905c69b:::2:head 2024-03-13T07:42:02.259 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : cursor()=5:f905c69b:::2:head expected=5:f905c69b:::2:head 2024-03-13T07:42:02.259 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:f905c69b:::2:head -> 2 2024-03-13T07:42:02.259 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : entry=2 expected=2 2024-03-13T07:42:02.260 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:e960b815:::9:head 2024-03-13T07:42:02.260 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : cursor()=5:e960b815:::9:head expected=5:e960b815:::9:head 2024-03-13T07:42:02.260 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:e960b815:::9:head -> 9 2024-03-13T07:42:02.260 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : entry=9 expected=9 2024-03-13T07:42:02.260 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:de5d7c5f:::12:head 2024-03-13T07:42:02.260 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : cursor()=5:de5d7c5f:::12:head expected=5:de5d7c5f:::12:head 2024-03-13T07:42:02.260 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:de5d7c5f:::12:head -> 12 2024-03-13T07:42:02.260 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : entry=12 expected=12 2024-03-13T07:42:02.260 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : seek to 5:5c6b0b28:::7:head 2024-03-13T07:42:02.260 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : cursor()=5:5c6b0b28:::7:head expected=5:5c6b0b28:::7:head 2024-03-13T07:42:02.260 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: > 5:5c6b0b28:::7:head -> 7 2024-03-13T07:42:02.260 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: : entry=7 expected=7 2024-03-13T07:42:02.260 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (8424 ms) 2024-03-13T07:42:02.260 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-03-13T07:42:02.260 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (175806 ms) 2024-03-13T07:42:02.261 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-03-13T07:42:02.261 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: split 0/5 -> MIN 5:33333333::::head 2024-03-13T07:42:02.261 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: split 1/5 -> 5:33333333::::head 5:66666666::::head 2024-03-13T07:42:02.261 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: split 2/5 -> 5:66666666::::head 5:99999999::::head 2024-03-13T07:42:02.261 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: split 3/5 -> 5:99999999::::head 5:cccccccc::::head 2024-03-13T07:42:02.261 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: split 4/5 -> 5:cccccccc::::head MAX 2024-03-13T07:42:02.261 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (19641 ms) 2024-03-13T07:42:02.261 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [----------] 7 tests from LibRadosList (351262 ms total) 2024-03-13T07:42:02.261 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 2024-03-13T07:42:02.261 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-03-13T07:42:02.261 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-03-13T07:42:02.261 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (3914 ms) 2024-03-13T07:42:02.261 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-03-13T07:42:02.261 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: myset foo1,foo2,foo3 2024-03-13T07:42:02.262 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: foo1 2024-03-13T07:42:02.262 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: foo2 2024-03-13T07:42:02.262 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: foo3 2024-03-13T07:42:02.262 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: myset foo1,foo4,foo5 2024-03-13T07:42:02.262 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: foo4 2024-03-13T07:42:02.262 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: foo5 2024-03-13T07:42:02.262 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: foo1 2024-03-13T07:42:02.262 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: myset foo6,foo7 2024-03-13T07:42:02.262 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: foo7 2024-03-13T07:42:02.262 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: foo6 2024-03-13T07:42:02.262 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-03-13T07:42:02.262 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: ns1:foo4 2024-03-13T07:42:02.262 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: ns1:foo5 2024-03-13T07:42:02.262 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: ns2:foo7 2024-03-13T07:42:02.263 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: ns2:foo6 2024-03-13T07:42:02.263 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: ns1:foo1 2024-03-13T07:42:02.263 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: :foo1 2024-03-13T07:42:02.263 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: :foo2 2024-03-13T07:42:02.263 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: :foo3 2024-03-13T07:42:02.263 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (4735 ms) 2024-03-13T07:42:02.263 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-03-13T07:42:02.263 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 1 0 2024-03-13T07:42:02.263 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 10 0 2024-03-13T07:42:02.263 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 13 0 2024-03-13T07:42:11.654 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 7 0 2024-03-13T07:42:11.654 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 14 0 2024-03-13T07:42:11.654 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 0 0 2024-03-13T07:42:11.654 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 15 0 2024-03-13T07:42:11.654 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 11 0 2024-03-13T07:42:11.654 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 5 0 2024-03-13T07:42:11.654 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 8 0 2024-03-13T07:42:11.654 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 6 0 2024-03-13T07:42:11.654 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 3 0 2024-03-13T07:42:11.654 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 4 0 2024-03-13T07:42:11.655 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 12 0 2024-03-13T07:42:11.655 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 9 0 2024-03-13T07:42:11.655 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 2 0 2024-03-13T07:42:11.655 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-03-13T07:42:11.655 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (6537 ms) 2024-03-13T07:42:11.655 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [----------] 3 tests from LibRadosListEC (15187 ms total) 2024-03-13T07:42:11.655 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 2024-03-13T07:42:11.655 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-03-13T07:42:11.655 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-03-13T07:42:11.655 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3312 ms) 2024-03-13T07:42:11.655 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [----------] 1 test from LibRadosListNP (3312 ms total) 2024-03-13T07:42:11.655 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: 2024-03-13T07:42:11.655 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [----------] Global test environment tear-down 2024-03-13T07:42:11.655 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [==========] 11 tests from 3 test suites ran. (379932 ms total) 2024-03-13T07:42:11.656 INFO:tasks.workunit.client.0.smithi132.stdout: api_list: [ PASSED ] 11 tests. 2024-03-13T07:42:17.159 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:08:41.049 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:17.165 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:08:41.056 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:17.221 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:08:41.092 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:17.228 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:08:41.099 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:17.242 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:08:41.131 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:17.247 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:08:41.136 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:17.247 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:08:41.137 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:17.288 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:08:41.158 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:17.317 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:08:41.208 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:29.435 INFO:tasks.workunit.client.0.smithi132.stdout: list: Running main() from gmock_main.cc 2024-03-13T07:42:29.435 INFO:tasks.workunit.client.0.smithi132.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-03-13T07:42:29.435 INFO:tasks.workunit.client.0.smithi132.stdout: list: [----------] Global test environment set-up. 2024-03-13T07:42:29.435 INFO:tasks.workunit.client.0.smithi132.stdout: list: [----------] 3 tests from NeoradosList 2024-03-13T07:42:29.436 INFO:tasks.workunit.client.0.smithi132.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-03-13T07:42:29.436 INFO:tasks.workunit.client.0.smithi132.stdout: list: [ OK ] NeoradosList.ListObjects (140230 ms) 2024-03-13T07:42:29.436 INFO:tasks.workunit.client.0.smithi132.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-03-13T07:42:29.436 INFO:tasks.workunit.client.0.smithi132.stdout: list: [ OK ] NeoradosList.ListObjectsNS (46187 ms) 2024-03-13T07:42:29.436 INFO:tasks.workunit.client.0.smithi132.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-03-13T07:42:29.436 INFO:tasks.workunit.client.0.smithi132.stdout: list: [ OK ] NeoradosList.ListObjectsMany (210882 ms) 2024-03-13T07:42:29.436 INFO:tasks.workunit.client.0.smithi132.stdout: list: [----------] 3 tests from NeoradosList (397299 ms total) 2024-03-13T07:42:29.436 INFO:tasks.workunit.client.0.smithi132.stdout: list: 2024-03-13T07:42:29.436 INFO:tasks.workunit.client.0.smithi132.stdout: list: [----------] Global test environment tear-down 2024-03-13T07:42:29.436 INFO:tasks.workunit.client.0.smithi132.stdout: list: [==========] 3 tests from 1 test suite ran. (397299 ms total) 2024-03-13T07:42:29.436 INFO:tasks.workunit.client.0.smithi132.stdout: list: [ PASSED ] 3 tests. 2024-03-13T07:42:37.208 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:09:01.098 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:37.214 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:09:01.105 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:37.268 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:09:01.159 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:37.269 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:09:01.160 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:37.271 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:09:01.161 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:37.276 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:09:01.146 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:37.282 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:09:01.153 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:37.333 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:09:01.223 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:37.333 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:09:01.204 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:50.028 INFO:tasks.workunit.client.0.smithi132.stdout: ec_list: Running main() from gmock_main.cc 2024-03-13T07:42:50.029 INFO:tasks.workunit.client.0.smithi132.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-03-13T07:42:50.029 INFO:tasks.workunit.client.0.smithi132.stdout: ec_list: [----------] Global test environment set-up. 2024-03-13T07:42:50.029 INFO:tasks.workunit.client.0.smithi132.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-03-13T07:42:50.029 INFO:tasks.workunit.client.0.smithi132.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-03-13T07:42:50.029 INFO:tasks.workunit.client.0.smithi132.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (59955 ms) 2024-03-13T07:42:50.029 INFO:tasks.workunit.client.0.smithi132.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-03-13T07:42:50.029 INFO:tasks.workunit.client.0.smithi132.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (109793 ms) 2024-03-13T07:42:50.029 INFO:tasks.workunit.client.0.smithi132.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-03-13T07:42:50.029 INFO:tasks.workunit.client.0.smithi132.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (248137 ms) 2024-03-13T07:42:50.029 INFO:tasks.workunit.client.0.smithi132.stdout: ec_list: [----------] 3 tests from NeoradosECList (417885 ms total) 2024-03-13T07:42:50.030 INFO:tasks.workunit.client.0.smithi132.stdout: ec_list: 2024-03-13T07:42:50.030 INFO:tasks.workunit.client.0.smithi132.stdout: ec_list: [----------] Global test environment tear-down 2024-03-13T07:42:50.030 INFO:tasks.workunit.client.0.smithi132.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (417885 ms total) 2024-03-13T07:42:50.030 INFO:tasks.workunit.client.0.smithi132.stdout: ec_list: [ PASSED ] 3 tests. 2024-03-13T07:42:55.785 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-03-13T07:42:55.785 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-03-13T07:42:55.785 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-03-13T07:42:55.785 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-03-13T07:42:55.785 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-03-13T07:42:55.785 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-13T07:42:55.785 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: 2024-03-13T07:42:55.785 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-03-13T07:42:55.786 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: seed 41026 2024-03-13T07:42:55.786 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-03-13T07:42:55.786 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (196 ms) 2024-03-13T07:42:55.786 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-03-13T07:42:55.786 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (54992 ms) 2024-03-13T07:42:55.786 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-03-13T07:42:55.786 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (24914 ms) 2024-03-13T07:42:55.786 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-03-13T07:42:55.786 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (31555 ms) 2024-03-13T07:42:55.786 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-03-13T07:42:55.786 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (2184 ms) 2024-03-13T07:42:55.786 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-03-13T07:42:55.786 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (6031 ms) 2024-03-13T07:42:55.787 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-03-13T07:42:55.787 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (868 ms) 2024-03-13T07:42:55.787 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-03-13T07:42:55.787 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (120 ms) 2024-03-13T07:42:55.787 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-03-13T07:42:55.787 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (78 ms) 2024-03-13T07:42:55.787 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-03-13T07:42:55.787 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (4446 ms) 2024-03-13T07:42:55.787 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-03-13T07:42:55.787 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (229 ms) 2024-03-13T07:42:55.787 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-03-13T07:42:55.787 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (138 ms) 2024-03-13T07:42:55.787 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-03-13T07:42:55.787 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (146 ms) 2024-03-13T07:42:55.787 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-03-13T07:42:55.788 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: osd_max_attr_size = 0 2024-03-13T07:42:55.788 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-03-13T07:42:55.788 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (190545 ms) 2024-03-13T07:42:55.788 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-03-13T07:42:55.788 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (14815 ms) 2024-03-13T07:42:55.788 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-03-13T07:42:55.788 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-03-13T07:42:55.788 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: done waiting, doing copies 2024-03-13T07:42:55.788 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: waiting for final deep scrubs... 2024-03-13T07:42:55.788 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: done waiting 2024-03-13T07:42:55.788 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (80238 ms) 2024-03-13T07:42:55.788 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-03-13T07:42:55.788 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (615 ms) 2024-03-13T07:42:55.789 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-03-13T07:42:55.789 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (70 ms) 2024-03-13T07:42:55.789 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-03-13T07:42:55.789 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (5782 ms) 2024-03-13T07:42:55.789 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-03-13T07:42:55.789 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-03-13T07:42:55.789 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-03-13T07:42:55.789 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-03-13T07:42:55.789 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-03-13T07:42:55.789 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (1 ms) 2024-03-13T07:42:57.550 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:09:21.441 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:57.556 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:09:21.447 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:57.560 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:09:21.450 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:57.560 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:09:21.450 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:57.615 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:09:21.486 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:57.616 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:09:21.487 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:57.637 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:09:21.528 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:57.678 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:09:21.549 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:42:57.712 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:09:21.603 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:12.218 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [----- write_operations: Running main() from gmock_main.cc 2024-03-13T07:43:12.218 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-03-13T07:43:12.218 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [----------] Global test environment set-up. 2024-03-13T07:43:12.219 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-03-13T07:43:12.219 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-03-13T07:43:12.219 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (34861 ms) 2024-03-13T07:43:12.219 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-03-13T07:43:12.219 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (31964 ms) 2024-03-13T07:43:12.219 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-03-13T07:43:12.219 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (69682 ms) 2024-03-13T07:43:12.219 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-03-13T07:43:12.219 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (46956 ms) 2024-03-13T07:43:12.219 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-03-13T07:43:12.219 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (13271 ms) 2024-03-13T07:43:12.219 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-03-13T07:43:12.219 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (13388 ms) 2024-03-13T07:43:12.219 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-03-13T07:43:12.219 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (229897 ms) 2024-03-13T07:43:12.220 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (440019 ms total) 2024-03-13T07:43:12.220 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: 2024-03-13T07:43:12.220 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [----------] Global test environment tear-down 2024-03-13T07:43:12.220 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (440020 ms total) 2024-03-13T07:43:12.220 INFO:tasks.workunit.client.0.smithi132.stdout: write_operations: [ PASSED ] 7 tests. 2024-03-13T07:43:17.479 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:09:41.369 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:17.482 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:09:41.372 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:17.531 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:09:41.402 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:17.533 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:09:41.404 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:17.543 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:09:41.434 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:17.548 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:09:41.438 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:17.560 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:09:41.451 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:17.579 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:09:41.450 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:17.625 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:09:41.515 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:17.970 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-03-13T07:43:17.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-03-13T07:43:17.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-03-13T07:43:17.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-03-13T07:43:17.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-03-13T07:43:17.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-03-13T07:43:17.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-03-13T07:43:17.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: 2024-03-13T07:43:17.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-03-13T07:43:17.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-03-13T07:43:17.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (23914 ms) 2024-03-13T07:43:17.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-03-13T07:43:17.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (31952 ms) 2024-03-13T07:43:17.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-03-13T07:43:17.971 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (81145 ms) 2024-03-13T07:43:17.972 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-03-13T07:43:17.972 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (49560 ms) 2024-03-13T07:43:17.972 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-03-13T07:43:17.972 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (10327 ms) 2024-03-13T07:43:17.972 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-03-13T07:43:17.972 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (11320 ms) 2024-03-13T07:43:17.972 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-03-13T07:43:17.972 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (237787 ms) 2024-03-13T07:43:17.972 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (446005 ms total) 2024-03-13T07:43:17.972 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: 2024-03-13T07:43:17.972 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-03-13T07:43:17.972 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (446005 ms total) 2024-03-13T07:43:17.972 INFO:tasks.workunit.client.0.smithi132.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-03-13T07:43:28.678 INFO:tasks.workunit.client.0.smithi132.stdout:-----] 22 tests from LibRadosMiscPP (417963 ms total) 2024-03-13T07:43:28.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: 2024-03-13T07:43:28.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-03-13T07:43:28.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-03-13T07:43:28.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (4102 ms) 2024-03-13T07:43:28.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (4102 ms total) 2024-03-13T07:43:28.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: 2024-03-13T07:43:28.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-03-13T07:43:28.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-03-13T07:43:28.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (1277 ms) 2024-03-13T07:43:28.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-03-13T07:43:28.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (43 ms) 2024-03-13T07:43:28.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (1321 ms total) 2024-03-13T07:43:28.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: 2024-03-13T07:43:28.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-03-13T07:43:28.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-03-13T07:43:28.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (782 ms) 2024-03-13T07:43:28.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-03-13T07:43:28.680 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (52 ms) 2024-03-13T07:43:28.680 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (834 ms total) 2024-03-13T07:43:28.680 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: 2024-03-13T07:43:28.680 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-03-13T07:43:28.680 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-03-13T07:43:28.680 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (552 ms) 2024-03-13T07:43:28.680 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-03-13T07:43:28.680 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (41 ms) 2024-03-13T07:43:28.680 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (593 ms total) 2024-03-13T07:43:28.680 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: 2024-03-13T07:43:28.680 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-03-13T07:43:28.680 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-03-13T07:43:28.680 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1726 ms) 2024-03-13T07:43:28.681 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1726 ms total) 2024-03-13T07:43:28.681 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: 2024-03-13T07:43:28.681 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-03-13T07:43:28.681 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (456907 ms total) 2024-03-13T07:43:28.681 INFO:tasks.workunit.client.0.smithi132.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-03-13T07:43:37.580 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:10:01.471 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:37.586 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:10:01.477 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:37.595 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:10:01.486 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:37.598 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:10:01.468 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:37.654 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:10:01.544 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:37.672 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:10:01.543 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:37.713 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:10:01.603 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:37.724 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:10:01.614 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:37.739 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:10:01.610 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:38.897 INFO:tasks.workunit.client.0.smithi132.stdout:eWrite 2024-03-13T07:43:38.897 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7809 ms) 2024-03-13T07:43:38.897 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-03-13T07:43:38.897 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (10516 ms) 2024-03-13T07:43:38.897 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-03-13T07:43:38.897 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (9522 ms) 2024-03-13T07:43:38.897 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-03-13T07:43:38.898 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (8155 ms) 2024-03-13T07:43:38.898 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-03-13T07:43:38.898 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (9292 ms) 2024-03-13T07:43:38.898 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-03-13T07:43:38.898 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (8196 ms) 2024-03-13T07:43:38.898 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-03-13T07:43:38.898 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7726 ms) 2024-03-13T07:43:38.898 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-03-13T07:43:38.898 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7402 ms) 2024-03-13T07:43:38.898 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-03-13T07:43:38.898 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (9380 ms) 2024-03-13T07:43:38.898 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-03-13T07:43:38.898 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (9050 ms) 2024-03-13T07:43:38.898 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-03-13T07:43:38.899 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (8369 ms) 2024-03-13T07:43:38.899 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-03-13T07:43:38.899 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (8230 ms) 2024-03-13T07:43:38.899 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-03-13T07:43:38.899 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (7850 ms) 2024-03-13T07:43:38.899 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-03-13T07:43:38.899 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7963 ms) 2024-03-13T07:43:38.899 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-03-13T07:43:38.899 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (6811 ms) 2024-03-13T07:43:38.899 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-03-13T07:43:38.899 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7248 ms) 2024-03-13T07:43:38.899 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (133520 ms total) 2024-03-13T07:43:38.899 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: 2024-03-13T07:43:38.899 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [----------] Global test environment tear-down 2024-03-13T07:43:38.900 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (467230 ms total) 2024-03-13T07:43:38.900 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio: [ PASSED ] 42 tests. 2024-03-13T07:43:56.012 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-03-13T07:43:56.012 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-03-13T07:43:56.013 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-03-13T07:43:56.013 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-03-13T07:43:56.013 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-03-13T07:43:56.013 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: notify 2024-03-13T07:43:56.013 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (462777 ms) 2024-03-13T07:43:56.013 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-03-13T07:43:56.013 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (411 ms) 2024-03-13T07:43:56.013 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (463188 ms total) 2024-03-13T07:43:56.013 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: 2024-03-13T07:43:56.013 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-03-13T07:43:56.013 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-03-13T07:43:56.013 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: notify 2024-03-13T07:43:56.013 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (616 ms) 2024-03-13T07:43:56.013 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-03-13T07:43:56.014 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: notify 2024-03-13T07:43:56.014 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (4363 ms) 2024-03-13T07:43:56.014 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-03-13T07:43:56.014 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (679 ms) 2024-03-13T07:43:56.014 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-03-13T07:43:56.014 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (229 ms) 2024-03-13T07:43:56.014 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-03-13T07:43:56.014 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: handle_notify cookie 94646550551264 notify_id 1709396983809 notifier_gid 19234 2024-03-13T07:43:56.014 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (154 ms) 2024-03-13T07:43:56.014 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-03-13T07:43:56.014 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: handle_notify cookie 94646550551264 notify_id 1709396983808 notifier_gid 19234 2024-03-13T07:43:56.014 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (163 ms) 2024-03-13T07:43:56.014 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-03-13T07:43:56.014 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: handle_notify cookie 94646550572416 notify_id 1709396983811 notifier_gid 19234 2024-03-13T07:43:56.015 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (157 ms) 2024-03-13T07:43:56.015 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-03-13T07:43:56.015 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: handle_notify cookie 94646550584464 notify_id 1709396983812 notifier_gid 19234 2024-03-13T07:43:56.015 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (121 ms) 2024-03-13T07:43:56.015 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-03-13T07:43:56.015 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: handle_notify cookie 94646550584464 notify_id 1709396983813 notifier_gid 19234 2024-03-13T07:43:56.015 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (121 ms) 2024-03-13T07:43:56.015 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-03-13T07:43:56.015 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: handle_notify cookie 94646550584464 notify_id 1709396983814 notifier_gid 19234 2024-03-13T07:43:56.015 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (134 ms) 2024-03-13T07:43:56.015 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-03-13T07:43:56.015 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: trying... 2024-03-13T07:43:56.015 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: handle_notify cookie 94646550584464 notify_id 1709396983809 notifier_gid 19234 2024-03-13T07:43:56.015 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: timed out 2024-03-13T07:43:57.701 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:10:21.592 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:57.708 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:10:21.598 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:57.753 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:10:21.643 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:57.760 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:10:21.651 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:57.762 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:10:21.633 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:57.765 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:10:21.636 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:57.767 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:10:21.658 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:57.806 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:10:21.676 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:57.809 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:10:21.700 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:58.492 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:10:22.363 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:58.493 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:10:22.364 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:58.501 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:10:22.372 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:58.554 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:10:22.445 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:58.555 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:10:22.446 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:58.564 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:10:22.455 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:58.592 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:10:22.482 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:58.593 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:10:22.484 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:43:58.603 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:10:22.493 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:12.768 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: flushing 2024-03-13T07:44:12.768 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: flushed 2024-03-13T07:44:12.768 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3088 ms) 2024-03-13T07:44:12.768 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-03-13T07:44:12.768 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: trying... 2024-03-13T07:44:12.768 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: handle_notify cookie 94646550586672 notify_id 1709396983810 notifier_gid 19234 2024-03-13T07:44:12.769 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: timed out 2024-03-13T07:44:12.769 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: flushing 2024-03-13T07:44:12.769 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: flushed 2024-03-13T07:44:12.769 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3134 ms) 2024-03-13T07:44:12.769 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-03-13T07:44:12.769 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: List watches 2024-03-13T07:44:12.769 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: notify2 2024-03-13T07:44:12.769 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: handle_notify cookie 94646550587872 notify_id 1713691951105 notifier_gid 19234 2024-03-13T07:44:12.769 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: notify2 done 2024-03-13T07:44:12.769 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: watch_check 2024-03-13T07:44:12.769 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: unwatch2 2024-03-13T07:44:12.769 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: flushing 2024-03-13T07:44:12.769 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: done 2024-03-13T07:44:12.769 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (4233 ms) 2024-03-13T07:44:12.770 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-03-13T07:44:12.770 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: List watches 2024-03-13T07:44:12.770 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: notify2 2024-03-13T07:44:12.770 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: handle_notify cookie 94646550587872 notify_id 1726576852994 notifier_gid 19234 2024-03-13T07:44:12.770 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: notify2 done 2024-03-13T07:44:12.770 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: watch_check 2024-03-13T07:44:12.770 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: unwatch2 2024-03-13T07:44:12.770 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: flushing 2024-03-13T07:44:12.770 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: done 2024-03-13T07:44:12.770 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3581 ms) 2024-03-13T07:44:12.770 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (20774 ms total) 2024-03-13T07:44:12.770 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: 2024-03-13T07:44:12.770 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-03-13T07:44:12.770 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (500886 ms total) 2024-03-13T07:44:12.771 INFO:tasks.workunit.client.0.smithi132.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-03-13T07:44:22.810 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:10:46.700 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:22.814 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:10:46.705 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:22.859 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:10:46.750 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:22.865 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:10:46.736 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:22.866 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:10:46.737 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:22.870 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:10:46.761 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:22.872 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:10:46.762 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:22.911 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:10:46.782 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:22.920 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:10:46.811 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:42.991 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:11:06.881 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:42.995 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:11:06.885 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:43.046 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:11:06.936 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:43.052 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:11:06.923 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:43.058 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:11:06.929 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:43.059 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:11:06.950 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:43.061 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:11:06.951 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:43.102 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:11:06.993 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:44:43.115 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:11:06.986 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:03.092 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:11:26.963 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:03.093 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:11:26.983 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:03.094 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:11:26.965 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:03.100 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:11:26.990 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:03.137 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:11:27.008 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:03.148 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:11:27.039 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:03.150 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:11:27.041 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:03.155 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:11:27.046 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:03.210 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:11:27.100 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:23.112 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:11:47.002 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:23.115 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:11:47.005 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:23.156 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:11:47.046 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:23.175 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:11:47.065 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:23.175 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:11:47.046 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:23.178 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:11:47.069 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:23.185 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:11:47.056 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:23.239 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:11:47.110 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:23.246 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:11:47.137 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:43.231 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:07.122 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:43.280 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:07.151 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:43.282 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:07.153 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:43.294 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:07.184 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:43.319 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:07.189 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:43.356 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:07.246 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:43.362 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:07.252 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:43.366 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:07.256 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:45:43.413 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:07.304 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:03.186 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:27.076 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:03.191 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:27.081 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:03.210 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:27.100 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:03.239 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:27.129 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:03.240 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:27.110 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:03.240 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:27.111 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:03.258 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:27.148 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:03.263 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:27.133 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:03.286 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:27.177 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:13.357 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:37.247 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:13.421 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:37.312 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:13.422 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:37.312 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:13.483 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:37.353 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:13.491 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:37.362 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:13.494 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:37.385 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:13.529 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:37.400 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:13.533 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:37.423 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:13.538 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:37.428 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:15.468 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:39.358 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:15.528 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:39.418 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:15.550 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:39.440 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:15.571 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:39.441 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:15.575 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:39.446 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:15.576 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:39.467 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:15.581 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:39.471 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:15.592 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:39.463 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:15.599 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:39.490 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:23.263 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:47.153 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:23.269 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:47.159 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:23.302 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:47.193 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:23.334 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:47.205 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:23.335 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:47.225 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:23.336 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:47.207 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:23.337 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:47.227 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:23.360 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:47.231 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:23.368 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:47.258 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:28.332 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:52.222 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:28.334 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:52.225 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:28.362 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:12:52.253 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:28.383 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:52.254 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:28.384 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:52.255 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:28.390 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:52.280 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:28.396 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:52.286 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:28.401 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:12:52.272 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:28.413 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:12:52.304 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:43.512 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:07.402 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:43.514 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:07.404 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:43.547 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:07.437 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:43.580 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:07.470 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:43.582 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:07.472 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:43.589 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:07.456 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:43.596 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:07.467 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:43.620 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:07.510 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:43.626 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:07.497 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:58.428 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:22.318 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:58.431 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:22.322 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:58.471 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:22.362 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:58.499 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:22.389 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:58.501 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:22.391 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:58.541 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:22.431 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:58.761 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:22.632 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:58.763 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:22.633 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:46:58.796 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:22.666 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:03.386 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:27.276 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:03.434 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:27.325 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:03.449 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:27.340 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:03.480 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:27.351 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:03.482 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:27.372 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:03.486 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:27.373 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:03.486 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:27.353 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:03.498 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:27.369 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:03.500 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:27.390 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:10.326 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-03-13T07:47:10.327 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-03-13T07:47:10.327 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-03-13T07:47:10.327 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: seed 41102 2024-03-13T07:47:10.327 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-03-13T07:47:10.327 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (35889 ms) 2024-03-13T07:47:10.327 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-03-13T07:47:10.327 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (64717 ms) 2024-03-13T07:47:10.327 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-03-13T07:47:10.327 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (12825 ms) 2024-03-13T07:47:10.327 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (113431 ms total) 2024-03-13T07:47:10.327 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: 2024-03-13T07:47:10.327 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-03-13T07:47:10.327 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-03-13T07:47:10.328 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (125185 ms) 2024-03-13T07:47:10.328 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-03-13T07:47:10.328 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (35947 ms) 2024-03-13T07:47:10.328 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-03-13T07:47:10.328 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (27977 ms) 2024-03-13T07:47:10.328 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-03-13T07:47:10.328 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: my_snaps [3] 2024-03-13T07:47:10.328 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: my_snaps [4,3] 2024-03-13T07:47:10.328 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: my_snaps [5,4,3] 2024-03-13T07:47:10.328 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-03-13T07:47:10.328 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: promoting some heads 2024-03-13T07:47:10.328 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: promoting from clones for snap 6 2024-03-13T07:47:10.328 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: promoting from clones for snap 5 2024-03-13T07:47:10.328 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: promoting from clones for snap 4 2024-03-13T07:47:10.329 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: promoting from clones for snap 3 2024-03-13T07:47:10.329 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: waiting for scrubs... 2024-03-13T07:47:10.329 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: done waiting 2024-03-13T07:47:10.329 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (172956 ms) 2024-03-13T07:47:10.329 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-03-13T07:47:10.329 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (11606 ms) 2024-03-13T07:47:10.329 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-03-13T07:47:10.329 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (10963 ms) 2024-03-13T07:47:10.329 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-03-13T07:47:10.329 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (9459 ms) 2024-03-13T07:47:10.329 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-03-13T07:47:10.329 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (10971 ms) 2024-03-13T07:47:10.329 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-03-13T07:47:10.330 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (11916 ms) 2024-03-13T07:47:10.330 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-03-13T07:47:10.330 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (10940 ms) 2024-03-13T07:47:10.330 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-03-13T07:47:10.330 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (11948 ms) 2024-03-13T07:47:10.330 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-03-13T07:47:10.330 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (38735 ms) 2024-03-13T07:47:10.330 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-03-13T07:47:10.330 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (10458 ms) 2024-03-13T07:47:10.330 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-03-13T07:47:10.330 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (11692 ms) 2024-03-13T07:47:10.330 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-03-13T07:47:10.330 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (16218 ms) 2024-03-13T07:47:10.330 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-03-13T07:47:10.331 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (9684 ms) 2024-03-13T07:47:10.331 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-03-13T07:47:10.331 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (10789 ms) 2024-03-13T07:47:13.603 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:37.493 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:13.656 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:37.546 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:13.699 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:37.590 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:13.714 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:37.604 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:13.719 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:37.609 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:13.744 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:37.635 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:13.774 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:37.644 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:13.775 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:37.646 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:13.800 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:37.670 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:23.506 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:47.396 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:23.573 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:47.463 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:23.603 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:47.494 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:23.635 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:47.526 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:23.637 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:47.528 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:23.663 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:47.553 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:23.667 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:47.538 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:23.669 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:47.540 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:23.725 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:47.595 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:28.674 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:52.564 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:28.677 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:52.567 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:28.700 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:13:52.591 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:28.724 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:52.614 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:28.725 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:52.596 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:28.725 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:52.616 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:28.728 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:52.598 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:28.740 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:13:52.630 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:28.744 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:13:52.614 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:32.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-03-13T07:47:32.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: hmm, no HitSet yet 2024-03-13T07:47:32.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: ok, hit_set contains 209:602f83fe:::foo:head 2024-03-13T07:47:32.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9629 ms) 2024-03-13T07:47:32.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-03-13T07:47:32.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg_num = 32 2024-03-13T07:47:32.797 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 0 ls 1710316047,0 2024-03-13T07:47:32.798 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 1 ls 1710316047,0 2024-03-13T07:47:32.798 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 2 ls 1710316047,0 2024-03-13T07:47:32.798 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 3 ls 1710316047,0 2024-03-13T07:47:32.798 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 4 ls 1710316047,0 2024-03-13T07:47:32.798 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 5 ls 1710316048,0 2024-03-13T07:47:32.798 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 6 ls 1710316048,0 2024-03-13T07:47:32.798 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 7 ls 1710316047,0 2024-03-13T07:47:32.798 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 8 ls 1710316047,0 2024-03-13T07:47:32.798 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 9 ls 1710316047,0 2024-03-13T07:47:32.798 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 10 ls 1710316047,0 2024-03-13T07:47:32.798 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 11 ls 1710316047,0 2024-03-13T07:47:32.798 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 12 ls 1710316047,0 2024-03-13T07:47:32.798 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 13 ls 1710316047,0 2024-03-13T07:47:32.799 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 14 ls 1710316047,0 2024-03-13T07:47:32.799 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 15 ls 1710316047,0 2024-03-13T07:47:32.799 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 16 ls 1710316047,0 2024-03-13T07:47:32.799 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 17 ls 1710316047,0 2024-03-13T07:47:32.799 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 18 ls 1710316048,0 2024-03-13T07:47:32.799 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 19 ls 1710316047,0 2024-03-13T07:47:32.799 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 20 ls 1710316047,0 2024-03-13T07:47:32.799 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 21 ls 1710316047,0 2024-03-13T07:47:32.799 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 22 ls 1710316047,0 2024-03-13T07:47:32.799 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 23 ls 1710316047,0 2024-03-13T07:47:32.799 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 24 ls 1710316047,0 2024-03-13T07:47:32.799 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 25 ls 1710316047,0 2024-03-13T07:47:32.799 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 26 ls 1710316047,0 2024-03-13T07:47:32.800 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 27 ls 1710316047,0 2024-03-13T07:47:32.800 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 28 ls 1710316048,0 2024-03-13T07:47:32.800 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 29 ls 1710316047,0 2024-03-13T07:47:32.800 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 30 ls 1710316047,0 2024-03-13T07:47:32.800 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg 31 ls 1710316047,0 2024-03-13T07:47:32.800 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: pg_num = 32 2024-03-13T07:47:32.800 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:6cac518f:::0:head 2024-03-13T07:47:32.800 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:02547ec2:::1:head 2024-03-13T07:47:32.800 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:f905c69b:::2:head 2024-03-13T07:47:32.800 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:cfc208b3:::3:head 2024-03-13T07:47:32.800 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:d83876eb:::4:head 2024-03-13T07:47:32.800 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:b29083e3:::5:head 2024-03-13T07:47:32.800 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:c4fdafeb:::6:head 2024-03-13T07:47:32.800 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:5c6b0b28:::7:head 2024-03-13T07:47:32.801 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:bd63b0f1:::8:head 2024-03-13T07:47:32.801 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:e960b815:::9:head 2024-03-13T07:47:32.801 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:52ea6a34:::10:head 2024-03-13T07:47:32.801 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:89d3ae78:::11:head 2024-03-13T07:47:32.801 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:de5d7c5f:::12:head 2024-03-13T07:47:32.801 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:566253c9:::13:head 2024-03-13T07:47:32.801 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:62a1935d:::14:head 2024-03-13T07:47:32.801 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:863748b0:::15:head 2024-03-13T07:47:32.801 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:3958e169:::16:head 2024-03-13T07:47:32.801 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:4d4dabf9:::17:head 2024-03-13T07:47:32.801 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:8391935d:::18:head 2024-03-13T07:47:32.801 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:28883081:::19:head 2024-03-13T07:47:32.801 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:69259c59:::20:head 2024-03-13T07:47:32.801 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:4bdb80b7:::21:head 2024-03-13T07:47:32.802 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:a11c5d71:::22:head 2024-03-13T07:47:32.802 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:271af37b:::23:head 2024-03-13T07:47:32.802 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:95b121be:::24:head 2024-03-13T07:47:32.802 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:58d1031b:::25:head 2024-03-13T07:47:32.802 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:0a050783:::26:head 2024-03-13T07:47:32.802 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:c709704c:::27:head 2024-03-13T07:47:32.802 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:cbe56eaf:::28:head 2024-03-13T07:47:32.802 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:86b4b162:::29:head 2024-03-13T07:47:32.802 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:70d89383:::30:head 2024-03-13T07:47:32.802 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:dd450c7c:::31:head 2024-03-13T07:47:32.802 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:6d5729b1:::32:head 2024-03-13T07:47:32.802 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:c388f3fb:::33:head 2024-03-13T07:47:32.803 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:56cfea31:::34:head 2024-03-13T07:47:32.803 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:9dbc1bf7:::35:head 2024-03-13T07:47:32.803 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:40b74ccd:::36:head 2024-03-13T07:47:32.803 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:4d5aaf42:::37:head 2024-03-13T07:47:32.803 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:920f362c:::38:head 2024-03-13T07:47:32.803 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:6cc53222:::39:head 2024-03-13T07:47:32.803 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:9cad833f:::40:head 2024-03-13T07:47:32.803 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:1ea84d41:::41:head 2024-03-13T07:47:32.803 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:c4480ef6:::42:head 2024-03-13T07:47:32.803 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:a694361e:::43:head 2024-03-13T07:47:32.803 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:d1bd33e9:::44:head 2024-03-13T07:47:32.803 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:ddc2cd5d:::45:head 2024-03-13T07:47:32.803 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:2b782207:::46:head 2024-03-13T07:47:32.803 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:7b187fca:::47:head 2024-03-13T07:47:32.804 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:90ecdf6f:::48:head 2024-03-13T07:47:32.804 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:a5ed95fe:::49:head 2024-03-13T07:47:32.804 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:ea0eaa55:::50:head 2024-03-13T07:47:32.804 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:f33ef17b:::51:head 2024-03-13T07:47:32.804 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:a0d1b2f6:::52:head 2024-03-13T07:47:32.804 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:60c5229e:::53:head 2024-03-13T07:47:32.804 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:edcbc575:::54:head 2024-03-13T07:47:32.804 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:102cf253:::55:head 2024-03-13T07:47:32.804 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:efb7fb0b:::56:head 2024-03-13T07:47:32.804 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:50d0a326:::57:head 2024-03-13T07:47:32.804 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:d4dc5daf:::58:head 2024-03-13T07:47:32.804 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:3a130462:::59:head 2024-03-13T07:47:32.804 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:ec87ed71:::60:head 2024-03-13T07:47:32.804 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:d5bc9454:::61:head 2024-03-13T07:47:32.804 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:3ddfe313:::62:head 2024-03-13T07:47:32.805 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:7c2816b9:::63:head 2024-03-13T07:47:32.805 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:47e00e4d:::64:head 2024-03-13T07:47:32.805 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:c6410c18:::65:head 2024-03-13T07:47:32.805 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:b48ed237:::66:head 2024-03-13T07:47:32.805 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:cd63ad31:::67:head 2024-03-13T07:47:32.805 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:b179e92b:::68:head 2024-03-13T07:47:32.805 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:0d9f741a:::69:head 2024-03-13T07:47:32.805 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:6d3352ae:::70:head 2024-03-13T07:47:32.805 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:c6d5c19e:::71:head 2024-03-13T07:47:32.805 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:bc4729c3:::72:head 2024-03-13T07:47:32.805 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:77e930b9:::73:head 2024-03-13T07:47:32.805 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:0abeecfd:::74:head 2024-03-13T07:47:32.805 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:b7c37e15:::75:head 2024-03-13T07:47:32.805 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:b6378398:::76:head 2024-03-13T07:47:32.805 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:02bd68de:::77:head 2024-03-13T07:47:32.806 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:cc795d2d:::78:head 2024-03-13T07:47:32.806 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:630d4fea:::79:head 2024-03-13T07:47:32.806 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:e0d29ef5:::80:head 2024-03-13T07:47:32.806 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:fd6f13d2:::81:head 2024-03-13T07:47:32.806 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:606461d5:::82:head 2024-03-13T07:47:32.806 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:eadbdc43:::83:head 2024-03-13T07:47:32.806 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:8761d0bb:::84:head 2024-03-13T07:47:32.806 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:9ef0186f:::85:head 2024-03-13T07:47:32.806 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:e0d41294:::86:head 2024-03-13T07:47:32.806 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:961de695:::87:head 2024-03-13T07:47:32.806 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:1423148f:::88:head 2024-03-13T07:47:32.806 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:633a8fa2:::89:head 2024-03-13T07:47:32.806 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:a8653809:::90:head 2024-03-13T07:47:32.806 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:3dac8b33:::91:head 2024-03-13T07:47:32.807 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:35aad435:::92:head 2024-03-13T07:47:32.807 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:f6dcc343:::93:head 2024-03-13T07:47:32.807 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:dbbdad87:::94:head 2024-03-13T07:47:32.807 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:1cb48ce0:::95:head 2024-03-13T07:47:32.807 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:03cd461c:::96:head 2024-03-13T07:47:32.807 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:17a4ea99:::97:head 2024-03-13T07:47:32.807 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:9993c9a7:::98:head 2024-03-13T07:47:32.807 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:6394211c:::99:head 2024-03-13T07:47:32.807 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:94c7ae57:::100:head 2024-03-13T07:47:32.807 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:6fdee5bb:::101:head 2024-03-13T07:47:32.807 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:9a477fd1:::102:head 2024-03-13T07:47:32.807 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:eb850916:::103:head 2024-03-13T07:47:32.807 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:affc56b9:::104:head 2024-03-13T07:47:32.807 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:b42dc814:::105:head 2024-03-13T07:47:32.807 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:f319f8f0:::106:head 2024-03-13T07:47:32.808 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:9a40b9de:::107:head 2024-03-13T07:47:32.808 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:8b524f28:::108:head 2024-03-13T07:47:32.808 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:e3de589f:::109:head 2024-03-13T07:47:32.808 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:90f90a5b:::110:head 2024-03-13T07:47:32.808 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:a7b4f1d7:::111:head 2024-03-13T07:47:32.808 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:af51766e:::112:head 2024-03-13T07:47:32.808 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:b6f90bd1:::113:head 2024-03-13T07:47:32.808 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:e0261208:::114:head 2024-03-13T07:47:32.808 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:c9569ef7:::115:head 2024-03-13T07:47:32.808 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:61bebe50:::116:head 2024-03-13T07:47:32.808 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:fe93412b:::117:head 2024-03-13T07:47:32.808 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:d3d38bee:::118:head 2024-03-13T07:47:32.808 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:3100ba0c:::119:head 2024-03-13T07:47:32.808 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:d0560ada:::120:head 2024-03-13T07:47:32.808 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:f0ea8b35:::121:head 2024-03-13T07:47:32.809 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:766f231a:::122:head 2024-03-13T07:47:32.809 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:a07a2582:::123:head 2024-03-13T07:47:32.809 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:bd7c6b3a:::124:head 2024-03-13T07:47:32.809 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:fb2ddaff:::125:head 2024-03-13T07:47:32.809 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:4408e1fe:::126:head 2024-03-13T07:47:32.809 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:ee1df7a7:::127:head 2024-03-13T07:47:32.809 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:c3002909:::128:head 2024-03-13T07:47:32.809 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:4f48ffa9:::129:head 2024-03-13T07:47:32.809 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:edf38733:::130:head 2024-03-13T07:47:32.809 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:c08425c0:::131:head 2024-03-13T07:47:32.809 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:5f902d98:::132:head 2024-03-13T07:47:32.809 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:41ea2c93:::133:head 2024-03-13T07:47:32.809 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:813cee13:::134:head 2024-03-13T07:47:32.809 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:0131818d:::135:head 2024-03-13T07:47:32.809 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:26ba5a85:::136:head 2024-03-13T07:47:32.810 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:381b8a5a:::137:head 2024-03-13T07:47:32.810 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:28797e47:::138:head 2024-03-13T07:47:32.810 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:bfca7f22:::139:head 2024-03-13T07:47:32.810 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:36807075:::140:head 2024-03-13T07:47:32.810 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:80b03975:::141:head 2024-03-13T07:47:32.810 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:5c15709b:::142:head 2024-03-13T07:47:32.810 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:f39ea15e:::143:head 2024-03-13T07:47:32.810 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:ea992956:::144:head 2024-03-13T07:47:32.810 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:48887b1c:::145:head 2024-03-13T07:47:32.810 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:9f24a9dd:::146:head 2024-03-13T07:47:32.810 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:987f100b:::147:head 2024-03-13T07:47:32.810 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:d2dd3581:::148:head 2024-03-13T07:47:32.810 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:7fed1808:::149:head 2024-03-13T07:47:32.810 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:c80b70e9:::150:head 2024-03-13T07:47:32.811 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:85ed90f9:::151:head 2024-03-13T07:47:32.811 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:36428b24:::152:head 2024-03-13T07:47:32.811 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:d044c34a:::153:head 2024-03-13T07:47:32.811 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:7c18bf58:::154:head 2024-03-13T07:47:32.811 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:d1c21232:::155:head 2024-03-13T07:47:32.811 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:a7a3c575:::156:head 2024-03-13T07:47:32.811 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:87da0633:::157:head 2024-03-13T07:47:32.811 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:d5ac3822:::158:head 2024-03-13T07:47:32.811 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:3f20522d:::159:head 2024-03-13T07:47:32.811 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:6ca26563:::160:head 2024-03-13T07:47:32.811 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:532ce135:::161:head 2024-03-13T07:47:32.811 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:c78863e6:::162:head 2024-03-13T07:47:43.671 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:07.561 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:43.680 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:07.570 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:43.708 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:07.598 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:43.735 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:07.605 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:43.736 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:07.607 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:43.742 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:07.633 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:43.745 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:07.635 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:43.763 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:07.633 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:43.773 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:07.663 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:48.901 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:12.791 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:48.906 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:12.796 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:48.928 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:12.818 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:48.968 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:12.858 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:48.971 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:12.861 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:48.975 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:12.846 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:48.979 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:12.850 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:48.990 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:12.881 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:47:49.006 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:12.876 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:04.079 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:27.969 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:04.082 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:27.972 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:04.129 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:28.018 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:04.159 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:28.029 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:04.162 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:28.052 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:04.164 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:28.055 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:04.168 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:28.039 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:04.204 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:28.094 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:04.211 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:28.082 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:05.056 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:28.927 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:05.057 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:28.928 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:05.067 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:28.937 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:05.124 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:29.014 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:05.125 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:29.016 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:05.141 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:29.031 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:05.159 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:29.049 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:05.161 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:29.051 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:05.173 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:29.064 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:29.160 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:53.050 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:29.163 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:53.053 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:29.210 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:53.080 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:29.211 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:53.082 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:29.235 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:53.126 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:29.237 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:53.127 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:29.238 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:53.129 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:29.257 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:53.127 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:29.303 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:53.193 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:34.184 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:58.074 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:34.188 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:58.078 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:34.203 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:14:58.094 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:34.231 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:58.121 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:34.236 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:58.126 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:34.236 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:58.106 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:34.243 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:58.113 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:34.249 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:14:58.139 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:34.254 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:14:58.125 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:39.827 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:7462ddf6:::163:head 2024-03-13T07:48:39.827 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:5d165639:::164:head 2024-03-13T07:48:39.827 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:f43765fc:::165:head 2024-03-13T07:48:39.827 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:b4c720e9:::166:head 2024-03-13T07:48:39.827 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:e694b040:::167:head 2024-03-13T07:48:39.827 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:afa38db2:::168:head 2024-03-13T07:48:39.827 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:77ba9f53:::169:head 2024-03-13T07:48:39.827 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:87495034:::170:head 2024-03-13T07:48:39.827 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:7c96bf0e:::171:head 2024-03-13T07:48:39.827 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:dbe346cc:::172:head 2024-03-13T07:48:39.827 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:e943ec24:::173:head 2024-03-13T07:48:39.828 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:f97a9c0c:::174:head 2024-03-13T07:48:39.828 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:6f26e74d:::175:head 2024-03-13T07:48:39.828 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:4f95e106:::176:head 2024-03-13T07:48:39.828 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:0e6f2f8f:::177:head 2024-03-13T07:48:39.828 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:05db05f1:::178:head 2024-03-13T07:48:39.828 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:38a78d66:::179:head 2024-03-13T07:48:39.828 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:d095610b:::180:head 2024-03-13T07:48:39.828 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:a1a9d709:::181:head 2024-03-13T07:48:39.828 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:1e5d39db:::182:head 2024-03-13T07:48:39.828 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:f7df4fb9:::183:head 2024-03-13T07:48:39.828 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:03a7f161:::184:head 2024-03-13T07:48:39.828 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:ba70721e:::185:head 2024-03-13T07:48:39.828 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:28e5662d:::186:head 2024-03-13T07:48:39.828 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:973d52de:::187:head 2024-03-13T07:48:39.828 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:4303eb1c:::188:head 2024-03-13T07:48:39.828 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:b990b48e:::189:head 2024-03-13T07:48:39.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:29b8165b:::190:head 2024-03-13T07:48:39.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:3547f197:::191:head 2024-03-13T07:48:39.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:7e260936:::192:head 2024-03-13T07:48:39.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:1abec7b1:::193:head 2024-03-13T07:48:39.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:10fdda93:::194:head 2024-03-13T07:48:39.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:15817eea:::195:head 2024-03-13T07:48:39.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:770bab57:::196:head 2024-03-13T07:48:39.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:ed9e13e7:::197:head 2024-03-13T07:48:39.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:71471a8f:::198:head 2024-03-13T07:48:39.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: checking for 210:10fb1d02:::199:head 2024-03-13T07:48:39.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (22392 ms) 2024-03-13T07:48:39.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-03-13T07:48:39.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316070,0 2024-03-13T07:48:39.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: first is 1710316070 2024-03-13T07:48:39.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316070,0 2024-03-13T07:48:39.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316070,0 2024-03-13T07:48:39.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316070,1710316072,1710316073,0 2024-03-13T07:48:39.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316070,1710316072,1710316073,0 2024-03-13T07:48:39.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316070,1710316072,1710316073,0 2024-03-13T07:48:39.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316070,1710316072,1710316073,1710316075,1710316076,0 2024-03-13T07:48:39.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316070,1710316072,1710316073,1710316075,1710316076,0 2024-03-13T07:48:39.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316070,1710316072,1710316073,1710316075,1710316076,0 2024-03-13T07:48:39.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316070,1710316072,1710316073,1710316075,1710316076,1710316078,1710316079,0 2024-03-13T07:48:39.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316070,1710316072,1710316073,1710316075,1710316076,1710316078,1710316079,0 2024-03-13T07:48:39.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316070,1710316072,1710316073,1710316075,1710316076,1710316078,1710316079,0 2024-03-13T07:48:39.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316073,1710316075,1710316076,1710316078,1710316079,1710316081,1710316082,0 2024-03-13T07:48:39.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: first now 1710316073, trimmed 2024-03-13T07:48:39.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (21255 ms) 2024-03-13T07:48:39.831 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-03-13T07:48:39.831 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: foo0 2024-03-13T07:48:39.831 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-03-13T07:48:39.831 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (15212 ms) 2024-03-13T07:48:39.831 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-03-13T07:48:49.243 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:13.133 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:49.248 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:13.138 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:49.281 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:13.171 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:49.304 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:13.174 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:49.306 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:13.196 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:49.306 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:13.177 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:49.308 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:13.198 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:49.337 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:13.207 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:49.339 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:13.229 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:50.030 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:13.901 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:50.031 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:13.902 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:50.039 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:13.910 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:50.089 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:13.980 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:50.090 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:13.981 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:50.100 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:13.990 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:50.118 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:14.008 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:50.119 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:14.010 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:48:50.132 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:14.022 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:04.334 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:28.224 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:04.366 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:28.254 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:04.388 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:28.259 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:04.397 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:28.287 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:04.398 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:28.289 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:04.424 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:28.314 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:04.442 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:28.313 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:04.451 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:28.341 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:04.473 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:28.343 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:09.424 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:33.314 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:09.427 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:33.317 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:09.478 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:33.368 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:09.480 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:33.371 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:09.488 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:33.358 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:09.488 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:33.378 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:09.489 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:33.359 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:09.499 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:33.390 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:09.506 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:33.377 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:19.352 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:43.241 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:19.355 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:43.246 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:19.384 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:43.274 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:19.408 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:43.278 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:19.409 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:43.299 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:19.411 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:43.281 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:19.412 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:43.302 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:19.436 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:43.306 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:19.438 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:43.328 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:29.395 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:53.285 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:29.407 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:53.298 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:29.416 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:53.287 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:29.422 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:53.310 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:29.425 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:53.315 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:29.480 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:53.351 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:29.510 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:53.380 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:29.510 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:53.400 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:29.519 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:53.410 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:34.449 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:58.339 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:34.499 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:58.389 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:34.531 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:15:58.421 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:34.544 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:58.415 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:34.550 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:58.439 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:34.552 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:58.443 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:34.569 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:58.439 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:34.569 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:15:58.460 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:34.584 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:15:58.455 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:49.545 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:16:13.435 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:49.598 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:16:13.488 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:49.631 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:16:13.521 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:49.647 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:16:13.518 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:49.649 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:16:13.520 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:49.657 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:16:13.547 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:49.659 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:16:13.549 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:49.679 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:16:13.549 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:49:49.681 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:16:13.571 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:09.658 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:16:33.548 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:09.663 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:16:33.553 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:09.693 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:16:33.584 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:09.717 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:16:33.587 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:09.722 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:16:33.592 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:09.730 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:16:33.620 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:09.733 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:16:33.623 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:09.752 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:16:33.622 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:09.768 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:16:33.658 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:24.608 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:16:48.498 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:24.611 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:16:48.501 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:24.613 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:16:48.483 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:24.616 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:16:48.486 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:24.641 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:16:48.511 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:24.645 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:16:48.535 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:24.702 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:16:48.592 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:24.703 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:16:48.593 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:24.729 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:16:48.619 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:29.716 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:16:53.606 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:29.718 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:16:53.608 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:29.726 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:16:53.596 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:29.728 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:16:53.618 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:29.730 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:16:53.601 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:29.732 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:16:53.622 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:29.787 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:16:53.677 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:29.799 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:16:53.669 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:29.808 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:16:53.699 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:39.868 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:03.758 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:39.869 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:03.759 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:39.894 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:03.784 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:39.915 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:03.805 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:39.922 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:03.812 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:39.935 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:03.806 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:39.937 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:03.807 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:39.948 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:03.838 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:39.961 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:03.832 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:54.777 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:18.667 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:54.782 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:18.673 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:54.809 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:18.699 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:54.828 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:18.699 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:54.830 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:18.701 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:54.832 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:18.722 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:54.833 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:18.724 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:54.858 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:18.729 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:54.862 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:18.752 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:55.542 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:19.433 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:55.543 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:19.434 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:55.544 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:19.414 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:55.544 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:19.415 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:55.551 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:19.442 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:55.552 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:19.423 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:55.589 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:19.479 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:55.590 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:19.481 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:55.598 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:19.489 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:59.492 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:23.382 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:59.494 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:23.385 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:59.539 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:23.429 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:59.540 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:23.411 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:59.542 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:23.412 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:59.545 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:23.435 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:59.549 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:23.438 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:59.556 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:23.426 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:50:59.563 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:23.453 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:14.992 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:38.881 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:14.993 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:38.883 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:15.018 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:38.908 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:15.056 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:38.946 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:15.057 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:38.948 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:15.062 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:38.932 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:15.063 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:38.934 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:15.092 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:38.982 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:15.095 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:38.965 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:29.973 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:53.862 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:29.976 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:53.867 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:30.005 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:53.896 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:30.039 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:53.910 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:30.041 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:53.931 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:30.042 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:53.913 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:30.048 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:53.939 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:30.074 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:53.944 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:30.086 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:53.976 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:35.064 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:58.954 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:35.067 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:58.957 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:35.084 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:17:58.974 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:35.139 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:59.009 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:35.144 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:59.015 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:35.163 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:17:59.034 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:35.211 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:59.100 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:35.215 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:59.105 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:35.239 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:17:59.129 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:45.046 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:18:08.936 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:45.096 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:18:08.966 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:45.097 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:18:08.967 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:45.104 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:18:08.994 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:45.124 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:18:08.994 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:45.130 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:18:09.020 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:45.173 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:18:09.063 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:45.176 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:18:09.067 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:45.205 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:18:09.095 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:54.928 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:18:18.818 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:54.974 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:18:18.864 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:54.988 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:18:18.878 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:55.021 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:18:18.911 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:55.022 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:18:18.912 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:55.032 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:18:18.902 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:55.033 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:18:18.903 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:55.036 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:18:18.926 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:51:55.046 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:18:18.917 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:03.936 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-03-13T07:52:03.917+0000 21279640 -1 snap_mapper.add_oid found existing snaps mapped on 228:04ee6edf:test-rados-api-smithi132-41102-80::foo:2, removing 2024-03-13T07:52:03.945 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T07:52:03.937+0000 23a7e640 -1 snap_mapper.add_oid found existing snaps mapped on 228:04ee6edf:test-rados-api-smithi132-41102-80::foo:2, removing 2024-03-13T07:52:05.137 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:18:29.027 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:05.196 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:18:29.086 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:05.219 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:18:29.109 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:05.254 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:18:29.125 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:05.257 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:18:29.147 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:05.259 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:18:29.129 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:05.260 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:18:29.151 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:05.274 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:18:29.145 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:05.276 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:18:29.166 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:20.290 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:18:44.180 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:20.292 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:18:44.182 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:20.333 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:18:44.223 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:20.350 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:18:44.221 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:20.352 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:18:44.223 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:20.357 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:18:44.247 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:20.360 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:18:44.250 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:20.394 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:18:44.284 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:20.394 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:18:44.265 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:35.169 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:18:59.059 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:35.172 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:18:59.062 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:35.206 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:18:59.095 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:35.219 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:18:59.090 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:35.221 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:18:59.091 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:35.224 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:18:59.115 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:35.226 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:18:59.116 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:35.254 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:18:59.124 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:35.260 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:18:59.150 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:39.493 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ProxyRead (21012 ms) 2024-03-13T07:52:39.493 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-03-13T07:52:39.493 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (26150 ms) 2024-03-13T07:52:39.493 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-03-13T07:52:39.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (4531 ms) 2024-03-13T07:52:39.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-03-13T07:52:39.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:52:39.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (4439 ms) 2024-03-13T07:52:39.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-03-13T07:52:39.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (5668 ms) 2024-03-13T07:52:39.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-03-13T07:52:39.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:52:39.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (5411 ms) 2024-03-13T07:52:39.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-03-13T07:52:39.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:52:39.494 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (5616 ms) 2024-03-13T07:52:39.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-03-13T07:52:39.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:52:39.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (41056 ms) 2024-03-13T07:52:39.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-03-13T07:52:39.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:52:39.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (18229 ms) 2024-03-13T07:52:39.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-03-13T07:52:39.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:52:39.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (5479 ms) 2024-03-13T07:52:39.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-03-13T07:52:39.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:52:39.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (5381 ms) 2024-03-13T07:52:39.495 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-03-13T07:52:39.496 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:52:39.496 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (28071 ms) 2024-03-13T07:52:39.496 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-03-13T07:52:39.496 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:52:39.496 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (16870 ms) 2024-03-13T07:52:39.496 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-03-13T07:52:39.496 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:52:39.496 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (6687 ms) 2024-03-13T07:52:39.496 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-03-13T07:52:39.496 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:52:39.496 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (5973 ms) 2024-03-13T07:52:39.496 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-03-13T07:52:39.496 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:52:39.497 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: waiting for scrubs... 2024-03-13T07:52:39.497 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: done waiting 2024-03-13T07:52:39.497 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25429 ms) 2024-03-13T07:52:39.497 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-03-13T07:52:39.497 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:52:39.497 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (12519 ms) 2024-03-13T07:52:39.497 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-03-13T07:52:39.497 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:52:39.497 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (10526 ms) 2024-03-13T07:52:39.497 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-03-13T07:52:39.497 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:52:39.497 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (11631 ms) 2024-03-13T07:52:40.327 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:04.217 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:40.329 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:04.219 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:40.352 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:04.242 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:40.385 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:04.255 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:40.391 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:04.262 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:40.413 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:04.303 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:40.414 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:04.284 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:40.423 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:04.313 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:40.447 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:04.337 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:55.237 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:19.127 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:55.240 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:19.130 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:55.275 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:19.165 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:55.291 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:19.162 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:55.292 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:19.182 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:55.292 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:19.163 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:55.293 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:19.183 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:55.328 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:19.218 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:52:55.334 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:19.205 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:00.249 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:24.139 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:00.253 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:24.143 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:00.265 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:24.156 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:00.298 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:24.168 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:00.301 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:24.171 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:00.301 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:24.191 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:00.303 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:24.193 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:00.313 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:24.183 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:00.315 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:24.205 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:10.446 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:34.336 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:10.448 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:34.339 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:10.477 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:34.367 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:10.505 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:34.376 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:10.508 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:34.378 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:10.538 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:34.428 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:10.539 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:34.410 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:10.540 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:34.430 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:10.573 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:34.464 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:20.325 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:44.214 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:20.328 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:44.219 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:20.345 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:44.235 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:20.376 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:44.266 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:20.377 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:44.247 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:20.377 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:44.268 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:20.378 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:44.248 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:20.393 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:44.283 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:20.394 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:44.265 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:25.486 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:49.376 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:25.493 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:49.383 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:25.493 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:49.364 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:25.498 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:49.368 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:25.521 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:49.411 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:25.526 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:49.396 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:25.556 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:49.446 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:25.559 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:49.449 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:25.578 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:49.468 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:30.364 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:54.254 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:30.367 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:54.258 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:30.381 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:19:54.271 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:30.416 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:54.286 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:30.419 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:54.289 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:30.423 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:54.313 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:30.424 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:54.314 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:30.429 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:19:54.300 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:30.435 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:19:54.325 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:45.421 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:09.310 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:45.426 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:09.316 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:45.450 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:09.340 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:45.474 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:09.364 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:45.474 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:09.344 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:45.475 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:09.346 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:45.476 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:09.366 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:45.498 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:09.368 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:45.499 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:09.389 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:49.342 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T07:53:49.327+0000 25281640 -1 snap_mapper.add_oid found existing snaps mapped on 90:31819bd0:test-rados-api-smithi132-41102-97::foo:21, removing 2024-03-13T07:53:49.370 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-03-13T07:53:49.362+0000 2968e640 -1 snap_mapper.add_oid found existing snaps mapped on 90:31819bd0:test-rados-api-smithi132-41102-97::foo:21, removing 2024-03-13T07:53:50.436 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:14.326 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:50.439 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:14.329 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:50.458 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:14.348 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:50.485 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:14.355 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:50.485 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:14.375 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:50.488 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:14.378 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:50.488 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:14.358 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:50.499 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:14.369 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:53:50.501 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:14.391 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:05.666 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:29.556 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:05.729 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:29.618 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:05.784 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:29.674 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:05.787 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:29.657 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:05.798 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:29.688 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:05.799 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:29.669 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:05.821 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:29.711 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:05.875 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:29.742 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:05.911 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:29.801 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:20.577 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:44.466 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:20.581 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:44.472 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:20.619 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:44.509 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:20.627 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:44.498 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:20.629 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:44.500 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:20.645 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:44.535 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:20.648 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:44.538 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:20.662 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:44.533 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:20.684 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:44.574 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:25.660 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:49.550 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:25.669 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:49.559 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:25.679 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:49.538 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:25.680 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:49.566 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:25.680 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:49.570 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:25.719 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:49.589 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:25.747 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:49.617 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:25.749 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:49.639 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:25.752 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:49.642 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:35.650 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:59.540 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:35.654 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:59.544 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:35.683 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:20:59.573 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:35.700 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:59.570 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:35.703 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:59.574 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:35.705 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:59.596 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:35.709 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:59.600 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:35.729 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:20:59.600 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:35.735 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:20:59.626 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:45.672 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:09.562 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:45.676 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:09.567 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:45.702 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:09.592 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:45.723 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:09.593 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:45.723 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:09.594 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:45.725 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:09.615 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:45.726 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:09.616 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:45.750 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:09.620 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:45.754 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:09.644 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:50.845 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:14.735 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:50.896 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:14.786 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:50.924 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:14.814 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:50.941 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:14.812 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:50.943 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:14.814 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:50.959 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:14.849 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:50.962 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:14.852 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:50.962 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:14.832 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:54:50.980 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:14.870 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:05.944 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:29.833 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:05.947 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:29.837 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:05.983 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:29.873 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:05.997 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:29.867 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:05.998 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:29.868 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:06.002 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:29.892 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:06.004 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:29.894 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:06.034 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:29.904 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:06.046 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:29.936 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:20.961 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:44.851 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:20.965 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:44.855 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:21.011 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:44.901 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:21.014 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:44.904 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:21.021 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:44.891 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:21.027 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:44.897 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:21.037 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:44.927 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:21.067 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:44.937 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:21.095 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:44.985 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:25.891 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:49.781 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:25.941 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:49.831 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:25.963 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:49.853 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:25.990 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:49.860 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:25.992 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:49.862 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:25.997 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:49.887 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:26.000 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:49.890 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:26.010 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:49.880 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:26.021 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:49.911 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:32.619 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:56.507 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:32.620 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:56.511 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:32.672 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:56.562 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:32.677 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:56.547 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:32.678 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:56.549 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:32.689 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:56.579 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:32.691 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:56.581 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:32.722 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:56.611 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:32.727 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:56.598 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:36.018 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:59.907 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:36.020 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:59.910 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:36.036 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:21:59.926 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:36.073 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:59.963 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:36.074 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:59.944 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:36.074 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:59.965 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:36.075 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:59.945 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:36.086 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:21:59.956 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:36.088 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:21:59.978 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:51.031 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:22:14.920 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:51.035 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:22:14.925 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:51.062 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:22:14.952 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:51.078 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:22:14.949 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:51.081 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:22:14.952 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:51.095 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:22:14.985 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:51.096 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:22:14.986 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:51.099 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:22:14.969 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:51.120 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:22:15.010 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:56.027 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:22:19.917 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:56.032 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:22:19.922 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:56.038 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:22:19.908 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:56.040 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:22:19.910 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:56.042 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:22:19.932 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:56.043 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:22:19.934 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:56.108 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:22:19.977 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:56.119 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:22:20.009 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:55:56.120 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:22:20.009 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:06.172 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:22:30.061 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:06.177 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:22:30.067 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:06.206 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:22:30.096 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:06.239 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:22:30.109 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:06.241 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:22:30.111 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:06.254 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:22:30.144 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:06.256 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:22:30.146 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:06.274 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:22:30.144 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:06.295 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:22:30.185 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:11.164 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:22:35.054 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:11.166 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:22:35.057 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:11.183 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:22:35.073 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:11.217 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:22:35.087 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:11.217 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:22:35.088 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:11.236 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:22:35.106 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:11.236 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:22:35.126 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:11.240 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:22:35.130 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:11.258 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:22:35.148 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:21.090 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:22:44.980 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:21.095 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:22:44.985 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:21.120 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:22:45.010 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:21.141 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:22:45.011 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:21.141 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:22:45.032 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:21.142 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:22:45.012 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:21.144 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:22:45.035 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:21.166 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:22:45.036 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:21.173 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:22:45.063 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:36.359 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:00.249 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:36.363 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:00.253 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:36.398 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:00.288 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:36.422 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:00.312 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:36.424 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:00.294 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:36.425 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:00.296 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:36.425 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:00.315 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:36.470 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:00.360 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:36.474 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:00.345 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:47.716 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-03-13T07:56:47.716 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:56:47.716 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (11928 ms) 2024-03-13T07:56:47.717 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-03-13T07:56:47.717 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:56:47.717 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (9358 ms) 2024-03-13T07:56:47.717 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-03-13T07:56:47.717 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:56:47.717 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (6664 ms) 2024-03-13T07:56:47.717 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-03-13T07:56:47.717 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:56:47.717 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (27975 ms) 2024-03-13T07:56:47.718 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-03-13T07:56:47.718 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:56:47.718 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (15454 ms) 2024-03-13T07:56:47.718 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-03-13T07:56:47.718 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12187 ms) 2024-03-13T07:56:47.718 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-03-13T07:56:47.718 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-03-13T07:56:47.718 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: 00000010 69 73 |is| 2024-03-13T07:56:47.718 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: 00000012 2024-03-13T07:56:47.718 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (13564 ms) 2024-03-13T07:56:47.718 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-03-13T07:56:47.719 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: require_osd_release = squid 2024-03-13T07:56:47.719 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (7997 ms) 2024-03-13T07:56:47.719 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (971738 ms total) 2024-03-13T07:56:47.719 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: 2024-03-13T07:56:47.719 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-03-13T07:56:47.719 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-03-13T07:56:47.719 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1634 ms) 2024-03-13T07:56:47.719 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-03-13T07:56:47.719 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11019 ms) 2024-03-13T07:56:47.719 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-03-13T07:56:47.719 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (20398 ms) 2024-03-13T07:56:47.719 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-03-13T07:56:47.720 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (192 ms) 2024-03-13T07:56:47.720 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (33243 ms total) 2024-03-13T07:56:47.720 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: 2024-03-13T07:56:47.720 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-03-13T07:56:47.720 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-03-13T07:56:47.720 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7950 ms) 2024-03-13T07:56:47.720 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-03-13T07:56:47.720 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (9998 ms) 2024-03-13T07:56:47.720 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-03-13T07:56:47.720 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: waiting for scrub... 2024-03-13T07:56:47.720 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: done waiting 2024-03-13T07:56:47.720 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (26787 ms) 2024-03-13T07:56:47.721 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-03-13T07:56:47.721 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (11963 ms) 2024-03-13T07:56:47.721 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-03-13T07:56:47.721 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (11308 ms) 2024-03-13T07:56:47.721 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-03-13T07:56:47.721 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (10168 ms) 2024-03-13T07:56:47.721 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-03-13T07:56:47.721 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (11935 ms) 2024-03-13T07:56:47.721 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-03-13T07:56:51.284 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:15.173 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:51.286 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:15.176 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:51.319 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:15.209 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:51.333 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:15.204 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:51.337 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:15.227 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:51.338 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:15.208 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:51.341 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:15.231 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:51.381 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:15.271 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:56:51.381 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:15.252 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:01.419 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:25.309 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:01.422 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:25.312 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:01.445 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:25.335 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:01.476 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:25.346 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:01.478 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:25.368 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:01.481 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:25.351 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:01.483 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:25.373 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:01.505 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:25.395 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:01.508 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:25.379 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:06.442 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:30.332 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:06.446 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:30.336 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:06.461 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:30.351 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:06.497 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:30.386 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:06.498 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:30.368 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:06.499 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:30.389 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:06.502 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:30.372 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:06.516 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:30.386 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:06.520 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:30.411 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:21.393 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:45.283 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:21.446 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:45.336 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:21.485 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:45.375 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:21.497 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:45.387 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:21.499 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:45.389 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:21.502 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:45.373 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:21.504 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:45.374 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:21.537 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:45.426 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:21.542 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:45.413 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:22.331 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:46.220 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:22.332 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:46.222 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:22.334 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:46.204 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:22.335 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:46.205 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:22.340 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:23:46.230 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:22.342 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:23:46.213 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:22.359 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:46.249 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:22.361 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:46.251 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:22.372 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:23:46.262 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:36.656 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:00.546 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:36.658 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:00.548 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:36.693 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:00.583 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:36.707 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:00.597 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:36.710 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:00.600 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:36.714 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:00.584 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:36.718 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:00.588 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:36.752 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:00.641 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:36.755 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:00.625 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:41.494 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:05.380 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:41.494 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:05.383 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:41.513 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:05.403 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:41.543 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:05.413 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:41.552 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:05.415 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:41.553 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:05.438 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:41.553 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:05.440 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:41.558 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:05.428 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:41.564 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:05.454 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:45.460 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:09.349 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:45.465 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:09.356 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:45.494 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:09.384 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:45.513 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:09.383 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:45.530 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:09.420 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:45.531 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:09.422 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:45.537 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:09.408 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:45.549 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:09.439 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:45.561 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:09.432 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:51.623 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:15.513 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:51.625 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:15.515 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:51.649 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:15.539 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:51.684 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:15.554 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:51.686 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:15.556 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:51.696 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:15.586 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:51.699 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:15.590 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:51.705 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:15.575 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:57:51.719 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:15.609 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:01.722 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:25.611 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:01.733 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:25.624 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:01.768 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:25.658 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:01.801 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:25.672 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:01.814 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:25.684 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:01.838 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:25.728 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:01.839 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:25.710 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:01.840 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:25.730 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:01.871 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:25.761 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:11.717 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:35.607 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:11.719 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:35.609 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:11.754 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:35.644 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:11.767 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:35.637 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:11.768 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:35.638 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:11.790 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:35.660 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:11.790 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:35.680 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:11.791 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:35.681 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:11.820 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:35.710 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:21.897 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:45.787 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:21.899 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:45.789 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:21.921 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:45.811 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:21.958 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:45.828 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:21.960 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:45.831 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:21.970 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:45.860 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:21.972 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:45.862 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:21.977 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:45.848 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:21.993 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:45.883 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:26.745 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:50.635 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:26.747 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:50.637 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:26.764 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:24:50.654 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:26.795 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:50.685 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:26.796 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:50.687 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:26.797 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:50.667 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:26.800 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:50.670 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:26.812 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:24:50.702 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:26.818 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:24:50.689 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:42.059 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:05.948 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:42.061 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:05.951 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:42.085 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:05.975 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:42.119 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:06.009 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:42.127 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:06.017 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:42.134 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:06.004 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:42.136 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:06.006 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:42.172 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:06.061 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:42.174 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:06.044 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:56.912 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:20.801 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:56.913 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:20.803 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:56.939 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:20.828 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:56.962 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:20.832 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:56.963 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:20.833 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:56.967 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:20.857 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:56.969 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:20.859 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:56.988 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:20.859 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:58:56.996 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:20.886 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:01.948 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:25.838 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:01.951 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:25.841 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:01.964 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:25.854 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:01.998 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:25.888 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:01.999 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:25.889 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:02.007 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:25.877 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:02.011 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:25.901 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:02.012 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:25.882 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:02.026 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:25.897 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:12.052 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:35.942 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:12.127 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:36.016 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:12.148 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:36.018 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:12.149 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:36.020 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:12.153 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:36.043 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:12.200 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:36.090 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:12.201 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:36.071 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:12.202 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:36.092 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:12.228 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:36.118 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:22.135 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:46.025 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:22.139 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:46.029 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:22.162 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:46.052 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:22.187 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:46.058 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:22.188 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:46.078 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:22.188 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:46.059 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:22.189 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:46.079 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:22.209 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:46.099 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:22.210 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:46.081 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:27.099 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:50.989 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:27.146 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:51.036 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:27.147 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:51.017 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:27.149 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:51.019 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:27.161 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:25:51.051 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:27.163 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:25:51.033 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:27.201 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:51.090 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:27.202 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:51.092 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:27.215 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:25:51.105 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:42.223 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:06.112 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:42.228 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:06.118 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:42.277 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:06.147 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:42.284 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:06.155 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:42.301 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:06.191 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:42.305 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:06.195 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:42.312 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:06.202 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:42.321 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:06.191 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:42.345 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:06.235 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:43.397 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:07.267 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:43.398 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:07.268 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:43.408 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:07.279 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:43.568 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:07.458 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:43.569 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:07.459 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:43.585 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:07.475 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:43.734 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:07.624 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:43.735 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:07.625 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:43.747 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:07.637 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:54.542 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (9665 ms) 2024-03-13T07:59:54.543 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-03-13T07:59:54.543 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (15816 ms) 2024-03-13T07:59:54.543 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-03-13T07:59:54.543 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (9966 ms) 2024-03-13T07:59:54.543 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-03-13T07:59:54.544 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (15094 ms) 2024-03-13T07:59:54.544 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-03-13T07:59:54.544 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (9773 ms) 2024-03-13T07:59:54.544 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-03-13T07:59:54.544 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (10570 ms) 2024-03-13T07:59:54.544 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-03-13T07:59:54.544 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: hmm, no HitSet yet 2024-03-13T07:59:54.544 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: ok, hit_set contains 260:602f83fe:::foo:head 2024-03-13T07:59:54.544 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (10041 ms) 2024-03-13T07:59:54.544 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-03-13T07:59:54.545 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316687,0 2024-03-13T07:59:54.545 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: first is 1710316687 2024-03-13T07:59:54.545 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316687,0 2024-03-13T07:59:54.545 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316687,0 2024-03-13T07:59:54.545 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316687,1710316689,1710316690,0 2024-03-13T07:59:54.545 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316687,1710316689,1710316690,0 2024-03-13T07:59:54.545 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316687,1710316689,1710316690,0 2024-03-13T07:59:54.545 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316687,1710316689,1710316690,1710316693,1710316694,0 2024-03-13T07:59:54.545 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316687,1710316689,1710316690,1710316693,1710316694,0 2024-03-13T07:59:54.545 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316687,1710316689,1710316690,1710316693,1710316694,0 2024-03-13T07:59:54.545 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316687,1710316689,1710316690,1710316693,1710316694,1710316696,1710316697,0 2024-03-13T07:59:54.545 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316687,1710316689,1710316690,1710316693,1710316694,1710316696,1710316697,0 2024-03-13T07:59:54.545 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316687,1710316689,1710316690,1710316693,1710316694,1710316696,1710316697,0 2024-03-13T07:59:54.545 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: got ls 1710316690,1710316693,1710316694,1710316696,1710316697,1710316699,1710316700,0 2024-03-13T07:59:54.546 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: first now 1710316690, trimmed 2024-03-13T07:59:54.546 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (22808 ms) 2024-03-13T07:59:54.546 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-03-13T07:59:54.546 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: foo0 2024-03-13T07:59:54.546 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-03-13T07:59:54.546 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (19626 ms) 2024-03-13T07:59:54.546 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-03-13T07:59:54.546 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (22102 ms) 2024-03-13T07:59:54.546 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-03-13T07:59:54.546 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (25779 ms) 2024-03-13T07:59:54.546 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-03-13T07:59:54.546 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (6497 ms) 2024-03-13T07:59:54.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-03-13T07:59:54.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (4660 ms) 2024-03-13T07:59:54.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-03-13T07:59:54.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (5959 ms) 2024-03-13T07:59:54.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-03-13T07:59:54.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (4591 ms) 2024-03-13T07:59:54.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (283056 ms total) 2024-03-13T07:59:54.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: 2024-03-13T07:59:54.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-03-13T07:59:54.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (1442718 ms total) 2024-03-13T07:59:54.547 INFO:tasks.workunit.client.0.smithi132.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-03-13T07:59:54.549 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T07:59:54.549 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40804 2024-03-13T07:59:54.549 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 40804 2024-03-13T07:59:54.549 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T07:59:54.549 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41012 2024-03-13T07:59:54.549 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41012 2024-03-13T07:59:54.550 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T07:59:54.550 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41508 2024-03-13T07:59:54.550 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41508 2024-03-13T07:59:54.550 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T07:59:54.550 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41255 2024-03-13T07:59:54.550 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41255 2024-03-13T07:59:54.550 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T07:59:54.550 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41086 2024-03-13T07:59:54.550 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41086 2024-03-13T07:59:54.550 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T07:59:54.550 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40886 2024-03-13T07:59:54.550 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 40886 2024-03-13T07:59:54.550 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T07:59:54.550 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41182 2024-03-13T07:59:54.551 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41182 2024-03-13T07:59:54.551 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T07:59:54.551 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41678 2024-03-13T07:59:54.551 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41678 2024-03-13T07:59:54.551 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T07:59:54.551 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41712 2024-03-13T07:59:54.551 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41712 2024-03-13T07:59:57.414 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:21.304 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:57.468 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:21.358 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:57.513 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:21.403 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:57.532 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:21.422 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:57.535 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:21.425 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:57.591 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:21.481 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:57.606 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:21.476 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:57.607 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:21.477 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:57.647 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:21.517 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:58.423 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:22.312 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:58.423 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:22.313 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:58.437 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:22.327 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:58.439 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:22.329 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:58.440 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:22.330 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:58.448 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:22.338 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:58.477 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:22.348 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:58.478 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:22.349 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T07:59:58.487 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:22.357 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:02.446 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:26.336 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:02.449 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:26.339 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:02.460 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:26.350 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:02.497 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:26.387 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:02.499 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:26.369 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:02.500 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:26.389 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:02.501 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:26.371 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:02.511 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:26.381 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:02.514 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:26.404 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:09.610 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:33.499 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:09.611 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:33.501 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:09.626 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:33.516 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:09.662 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:33.552 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:09.662 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:33.533 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:09.665 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:33.535 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:09.666 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:33.556 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:09.679 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:33.549 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:09.681 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:33.571 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:17.476 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:41.366 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:17.481 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:41.371 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:17.492 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:41.382 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:17.526 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:41.394 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:17.526 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:41.395 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:17.529 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:41.419 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:17.531 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:41.421 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:17.536 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:41.406 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:17.546 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:41.436 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:27.500 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:51.389 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:27.504 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:51.394 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:27.516 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:26:51.406 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:27.548 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:51.419 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:27.549 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:51.420 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:27.551 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:51.441 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:27.553 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:51.443 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:27.562 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:26:51.432 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:27.566 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:26:51.456 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:37.523 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:01.413 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:37.527 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:01.417 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:37.550 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:01.439 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:37.572 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:01.442 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:37.573 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:01.444 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:37.583 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:01.473 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:37.584 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:01.474 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:37.585 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:01.455 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:37.596 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:01.486 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:47.546 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:11.436 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:47.552 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:11.442 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:47.571 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:11.461 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:47.595 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:11.465 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:47.597 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:11.467 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:47.601 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:11.491 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:47.607 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:11.497 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:47.611 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:11.481 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:47.619 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:11.509 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:57.567 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:21.457 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:57.576 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:21.466 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:57.590 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:21.480 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:57.620 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:21.490 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:57.623 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:21.493 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:57.640 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:21.511 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:57.642 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:21.532 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:57.646 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:21.536 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:00:57.661 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:21.551 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:07.600 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:31.490 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:07.603 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:31.493 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:07.617 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:31.507 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:07.651 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:31.521 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:07.654 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:31.524 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:07.667 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:31.537 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:07.677 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:31.567 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:07.678 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:31.568 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:07.690 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:31.580 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:17.620 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:41.510 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:17.625 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:41.515 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:17.637 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:41.527 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:17.667 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:41.537 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:17.672 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:41.542 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:17.673 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:41.562 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:17.674 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:41.564 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:17.685 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:41.555 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:17.686 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:41.576 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:27.638 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:51.528 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:27.644 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:51.534 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:27.662 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:27:51.552 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:27.687 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:51.557 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:27.691 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:51.561 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:27.697 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:51.587 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:27.699 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:51.589 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:27.702 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:27:51.573 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:27.711 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:27:51.601 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:37.659 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:01.548 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:37.664 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:01.554 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:37.677 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:01.567 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:37.712 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:01.602 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:37.713 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:01.583 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:37.713 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:01.584 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:37.714 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:01.604 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:37.725 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:01.595 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:37.726 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:01.616 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:47.691 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:11.581 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:47.694 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:11.584 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:47.707 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:11.597 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:47.743 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:11.632 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:47.743 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:11.613 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:47.744 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:11.634 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:47.744 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:11.614 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:47.756 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:11.645 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:47.756 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:11.626 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:57.753 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:21.642 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:57.755 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:21.645 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:57.771 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:21.661 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:57.801 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:21.691 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:57.802 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:21.672 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:57.803 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:21.673 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:57.805 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:21.695 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:57.816 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:21.686 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:01:57.819 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:21.709 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:07.748 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:31.637 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:07.755 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:31.644 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:07.766 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:31.656 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:07.795 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:31.665 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:07.796 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:31.666 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:07.798 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:31.688 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:07.799 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:31.689 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:07.808 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:31.678 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:07.812 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:31.702 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:17.778 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:41.668 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:17.783 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:41.673 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:17.795 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:41.685 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:17.828 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:41.698 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:17.829 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:41.700 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:17.830 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:41.720 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:17.832 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:41.722 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:17.841 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:41.711 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:17.844 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:41.734 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:27.794 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:51.683 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:27.800 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:51.690 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:27.812 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:28:51.702 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:27.846 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:51.716 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:27.847 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:51.718 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:27.859 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:28:51.729 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:27.870 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:51.760 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:27.871 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:51.761 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:27.884 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:28:51.774 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:37.822 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:01.711 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:37.825 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:01.715 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:37.846 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:01.736 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:37.871 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:01.742 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:37.876 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:01.746 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:37.878 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:01.768 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:37.881 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:01.771 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:37.889 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:01.759 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:37.899 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:01.788 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:47.839 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:11.729 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:47.845 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:11.735 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:47.857 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:11.746 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:47.890 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:11.761 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:47.892 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:11.762 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:47.898 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:11.788 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:47.901 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:11.791 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:47.904 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:11.774 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:47.913 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:11.803 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:57.859 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:21.748 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:57.864 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:21.754 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:57.877 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:21.767 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:57.908 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:21.778 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:57.909 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:21.780 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:57.913 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:21.803 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:57.914 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:21.805 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:57.921 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:21.792 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:02:57.927 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:21.817 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:07.887 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:31.777 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:07.889 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:31.779 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:07.901 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:31.791 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:07.930 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:31.800 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:07.933 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:31.803 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:07.936 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:31.826 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:07.937 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:31.827 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:07.945 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:31.815 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:07.950 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:31.840 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:17.988 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:41.877 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:17.992 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:41.882 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:18.014 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:41.904 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:18.036 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:41.906 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:18.037 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:41.907 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:18.044 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:41.934 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:18.048 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:41.938 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:18.052 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:41.922 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:18.064 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:41.954 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:27.973 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:51.863 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:27.976 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:51.866 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:27.989 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:29:51.879 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:28.026 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:51.916 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:28.027 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:51.897 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:28.028 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:51.918 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:28.028 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:51.899 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:28.039 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:29:51.929 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:28.040 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:29:51.910 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:37.997 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:01.886 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:38.000 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:01.890 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:38.013 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:01.903 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:38.049 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:01.919 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:38.050 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:01.920 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:38.062 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:01.933 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:38.070 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:01.960 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:38.072 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:01.962 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:38.085 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:01.975 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:48.017 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:11.907 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:48.021 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:11.911 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:48.037 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:11.927 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:48.067 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:11.937 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:48.068 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:11.938 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:48.069 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:11.959 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:48.071 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:11.961 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:48.080 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:11.951 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:48.084 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:11.974 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:56.234 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:20.124 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:56.239 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:20.129 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:56.254 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:20.144 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:56.287 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:20.157 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:56.288 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:20.158 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:56.292 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:20.182 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:56.294 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:20.184 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:56.303 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:20.173 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:56.310 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:20.200 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:58.043 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:21.933 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:58.046 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:21.936 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:58.054 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:21.944 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:58.089 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:21.960 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:58.090 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:21.961 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:58.098 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:21.968 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:58.117 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:22.006 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:58.118 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:22.008 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:03:58.126 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:22.016 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:13.076 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:36.966 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:13.081 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:36.971 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:13.098 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:36.987 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:13.127 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:36.997 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:13.128 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:36.999 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:13.144 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:37.014 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:13.151 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:37.041 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:13.153 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:37.043 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:13.169 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:37.059 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:18.044 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:41.934 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:18.093 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:41.983 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:18.113 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:42.002 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:18.132 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:42.002 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:18.136 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:42.007 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:18.148 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:42.038 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:18.149 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:42.039 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:18.151 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:42.022 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:18.170 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:42.060 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:33.125 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:57.015 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:33.127 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:57.017 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:33.143 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:30:57.033 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:33.173 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:57.063 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:33.175 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:57.065 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:33.178 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:57.048 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:33.179 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:57.049 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:33.192 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:30:57.082 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:33.196 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:30:57.066 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.220 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:02.110 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.226 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:02.116 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.244 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:02.134 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.283 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:02.153 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.284 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:02.174 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.286 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:02.156 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.287 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:02.177 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.302 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:02.172 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.306 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:02.193 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.896 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:02.766 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.897 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:02.767 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.900 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:02.790 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.901 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:02.791 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.906 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:02.776 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.910 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:02.800 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.941 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:02.831 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.942 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:02.832 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:38.955 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:02.845 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:53.198 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:17.088 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:53.248 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:17.138 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:53.263 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:17.153 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:53.293 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:17.163 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:53.294 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:17.164 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:53.296 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:17.186 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:53.298 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:17.188 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:53.309 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:17.179 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:53.323 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:17.213 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:58.257 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:22.147 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:58.259 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:22.149 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:58.273 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:22.163 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:58.304 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:22.174 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:58.306 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:22.196 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:58.307 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:22.178 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:58.309 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:22.199 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:58.318 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:22.188 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:04:58.323 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:22.213 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:13.294 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:37.183 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:13.296 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:37.186 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:13.313 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:37.202 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:13.341 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:37.231 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:13.344 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:37.234 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:13.347 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:37.217 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:13.348 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:37.218 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:13.361 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:37.251 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:13.364 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:37.235 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:18.297 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:42.187 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:18.302 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:42.192 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:18.313 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:42.203 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:18.349 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:42.238 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:18.349 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:42.218 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:18.350 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:42.239 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:18.350 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:42.220 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:18.360 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:42.230 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:18.360 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:42.250 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:33.335 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:57.224 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:33.337 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:57.227 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:33.353 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:31:57.242 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:33.382 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:57.272 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:33.383 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:57.273 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:33.385 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:57.255 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:33.388 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:57.258 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:33.399 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:31:57.289 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:33.403 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:31:57.274 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:38.347 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:02.237 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:38.351 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:02.241 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:38.362 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:02.252 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:38.402 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:02.272 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:38.403 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:02.293 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:38.403 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:02.274 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:38.404 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:02.294 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:38.414 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:02.284 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:38.415 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:02.305 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:53.377 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:17.267 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:53.381 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:17.271 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:53.404 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:17.293 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:53.428 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:17.298 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:53.429 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:17.299 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:53.433 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:17.323 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:53.434 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:17.324 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:53.445 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:17.315 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:53.450 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:17.340 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:58.389 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:22.279 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:58.456 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:22.346 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:58.475 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:22.365 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:58.501 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:22.371 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:58.507 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:22.377 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:58.514 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:22.404 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:58.519 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:22.390 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:58.521 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:22.410 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:05:58.538 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:22.427 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:04.569 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: Running main() from gmock_main.cc 2024-03-13T08:06:04.570 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-03-13T08:06:04.570 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: [----------] Global test environment set-up. 2024-03-13T08:06:04.570 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-03-13T08:06:04.570 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-03-13T08:06:04.570 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: handle_notify cookie 94401207332352 notify_id 4329327034371 notifier_gid 4478 2024-03-13T08:06:04.570 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (1804788 ms) 2024-03-13T08:06:04.570 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-03-13T08:06:04.570 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: Trying... 2024-03-13T08:06:04.570 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: handle_notify cookie 94401220046864 notify_id 4342211936259 notifier_gid 18611 2024-03-13T08:06:04.570 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: Waiting for 3.000000000s 2024-03-13T08:06:04.570 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: Timed out. 2024-03-13T08:06:04.571 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: Flushing... 2024-03-13T08:06:04.571 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: Flushed... 2024-03-13T08:06:04.571 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (7595 ms) 2024-03-13T08:06:04.571 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (1812383 ms total) 2024-03-13T08:06:04.571 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: 2024-03-13T08:06:04.571 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: [----------] Global test environment tear-down 2024-03-13T08:06:04.571 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (1812383 ms total) 2024-03-13T08:06:04.571 INFO:tasks.workunit.client.0.smithi132.stdout: watch_notify: [ PASSED ] 2 tests. 2024-03-13T08:06:13.580 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:37.470 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:13.632 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:37.521 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:13.653 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:37.542 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:13.679 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:37.569 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:13.680 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:37.570 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:13.684 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:37.554 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:13.685 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:37.555 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:13.705 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:37.575 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:13.707 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:37.597 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:21.213 INFO:tasks.workunit.client.0.smithi132.stdout: ec_io: Running main() from gmock_main.cc 2024-03-13T08:06:21.213 INFO:tasks.workunit.client.0.smithi132.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-03-13T08:06:21.213 INFO:tasks.workunit.client.0.smithi132.stdout: ec_io: [----------] Global test environment set-up. 2024-03-13T08:06:21.213 INFO:tasks.workunit.client.0.smithi132.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-03-13T08:06:21.213 INFO:tasks.workunit.client.0.smithi132.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-03-13T08:06:21.213 INFO:tasks.workunit.client.0.smithi132.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (11461 ms) 2024-03-13T08:06:21.214 INFO:tasks.workunit.client.0.smithi132.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-03-13T08:06:21.214 INFO:tasks.workunit.client.0.smithi132.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (1817629 ms) 2024-03-13T08:06:21.214 INFO:tasks.workunit.client.0.smithi132.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (1829090 ms total) 2024-03-13T08:06:21.214 INFO:tasks.workunit.client.0.smithi132.stdout: ec_io: 2024-03-13T08:06:21.214 INFO:tasks.workunit.client.0.smithi132.stdout: ec_io: [----------] Global test environment tear-down 2024-03-13T08:06:21.214 INFO:tasks.workunit.client.0.smithi132.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (1829090 ms total) 2024-03-13T08:06:21.214 INFO:tasks.workunit.client.0.smithi132.stdout: ec_io: [ PASSED ] 2 tests. 2024-03-13T08:06:21.215 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:06:21.215 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40905 2024-03-13T08:06:21.215 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 40905 2024-03-13T08:06:21.215 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:06:21.216 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41537 2024-03-13T08:06:21.216 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41537 2024-03-13T08:06:21.216 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:06:21.216 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41400 2024-03-13T08:06:21.216 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41400 2024-03-13T08:06:21.216 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:06:21.216 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40776 2024-03-13T08:06:21.216 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 40776 2024-03-13T08:06:21.216 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:06:21.216 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40854 2024-03-13T08:06:21.216 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 40854 2024-03-13T08:06:21.216 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:06:21.216 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41418 2024-03-13T08:06:21.216 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41418 2024-03-13T08:06:21.217 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:06:21.217 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40767 2024-03-13T08:06:21.217 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 40767 2024-03-13T08:06:25.677 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: Running main() from gmock_main.cc 2024-03-13T08:06:25.677 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-03-13T08:06:25.677 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: [----------] Global test environment set-up. 2024-03-13T08:06:25.677 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-03-13T08:06:25.677 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-03-13T08:06:25.677 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (968 ms) 2024-03-13T08:06:25.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-03-13T08:06:25.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (154 ms) 2024-03-13T08:06:25.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-03-13T08:06:25.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (1827141 ms) 2024-03-13T08:06:25.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-03-13T08:06:25.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:19.159875+0000 mon.b [INF] from='client.4539 172.21.15.132:0/1563691866' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi132-41858-2"}]: dispatch 2024-03-13T08:06:25.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:19.175006+0000 mon.a [INF] from='client.4539 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi132-41858-2"}]: dispatch 2024-03-13T08:06:25.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:20.058588+0000 mon.a [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-13T08:06:25.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:20.062376+0000 mon.a [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-13T08:06:25.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:20.076046+0000 mon.a [INF] from='client.4539 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi132-41858-2"}]': finished 2024-03-13T08:06:25.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:20.166280+0000 mon.b [INF] from='client.4539 172.21.15.132:0/1563691866' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi132-41858-2"}]: dispatch 2024-03-13T08:06:25.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:20.187584+0000 mon.a [INF] from='client.4539 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi132-41858-2"}]: dispatch 2024-03-13T08:06:25.678 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:20.304244+0000 mon.c [INF] from='client.? 172.21.15.132:0/3843086167' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-13T08:06:25.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:20.347197+0000 client.admin [INF] onexx 2024-03-13T08:06:25.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:20.355559+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-13T08:06:25.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:21.097054+0000 mon.a [INF] from='client.4539 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi132-41858-2"}]': finished 2024-03-13T08:06:25.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:22.401674+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-13T08:06:25.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:22.491798+0000 mon.b [INF] from='client.? 172.21.15.132:0/1120401329' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi132-40848-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-13T08:06:25.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:22.520882+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi132-40848-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-13T08:06:25.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:23.481005+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi132-40848-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-13T08:06:25.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:24.496284+0000 mon.c [INF] from='client.? 172.21.15.132:0/3843086167' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-13T08:06:25.679 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:24.590227+0000 client.admin [INF] threexx 2024-03-13T08:06:28.852 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:52.741 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:28.854 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:52.744 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:28.864 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: got: 2024-03-13T08:06:24.595225+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-13T08:06:28.864 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (8700 ms) 2024-03-13T08:06:28.864 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (1836963 ms total) 2024-03-13T08:06:28.864 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: 2024-03-13T08:06:28.864 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: [----------] Global test environment tear-down 2024-03-13T08:06:28.864 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (1836963 ms total) 2024-03-13T08:06:28.864 INFO:tasks.workunit.client.0.smithi132.stdout: api_cmd: [ PASSED ] 4 tests. 2024-03-13T08:06:28.885 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:52.775 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:28.918 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:52.808 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:28.924 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:52.795 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:28.925 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:52.815 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:28.930 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:52.800 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:28.961 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:52.831 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:28.962 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:52.851 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:33.874 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:57.763 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:33.933 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:57.803 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:33.937 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:57.808 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:33.940 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:57.830 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:33.960 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:32:57.831 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:33.971 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:32:57.861 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:34.045 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:57.935 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:34.056 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:57.946 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:34.090 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:32:57.980 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:44.064 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:07.954 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:44.069 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:07.959 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:44.104 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:07.994 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:44.134 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:08.004 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:44.135 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:08.005 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:44.142 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:08.032 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:44.149 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:08.039 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:44.167 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:08.037 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:44.189 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:08.079 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:53.832 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:17.722 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:53.881 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:17.771 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:53.914 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:17.804 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:53.927 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:17.797 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:53.928 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:17.799 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:53.940 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:17.830 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:53.943 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:17.833 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:53.954 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:17.824 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:53.967 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:17.857 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:57.807 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:21.697 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:57.810 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:21.700 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:57.824 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:21.694 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:57.893 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:21.763 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:57.894 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:21.784 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:57.899 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:21.789 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:57.911 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:21.801 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:57.932 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:21.802 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:57.942 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:21.832 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:06:58.978 INFO:tasks.workunit.client.0.smithi132.stdout: misc: Running main() from gmock_main.cc 2024-03-13T08:06:58.978 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-03-13T08:06:58.978 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [----------] Global test environment set-up. 2024-03-13T08:06:58.978 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-03-13T08:06:58.978 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-03-13T08:06:58.978 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ OK ] NeoRadosMisc.Version (5757 ms) 2024-03-13T08:06:58.978 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-03-13T08:06:58.978 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (4649 ms) 2024-03-13T08:06:58.978 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-03-13T08:06:58.979 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ OK ] NeoRadosMisc.LongName (1827356 ms) 2024-03-13T08:06:58.979 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-03-13T08:06:58.979 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (3330 ms) 2024-03-13T08:06:58.979 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-03-13T08:06:58.979 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (2707 ms) 2024-03-13T08:06:58.979 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-03-13T08:06:58.979 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (3419 ms) 2024-03-13T08:06:58.979 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-03-13T08:06:58.979 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ OK ] NeoRadosMisc.Exec (3348 ms) 2024-03-13T08:06:58.979 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-03-13T08:06:58.979 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (3536 ms) 2024-03-13T08:06:58.979 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-03-13T08:06:58.979 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (3438 ms) 2024-03-13T08:06:58.979 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-03-13T08:06:58.979 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ OK ] NeoRadosMisc.BigObject (3370 ms) 2024-03-13T08:06:58.979 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-03-13T08:06:58.980 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (2516 ms) 2024-03-13T08:06:58.980 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-03-13T08:06:58.980 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (3386 ms) 2024-03-13T08:06:58.980 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [----------] 12 tests from NeoRadosMisc (1866812 ms total) 2024-03-13T08:06:58.980 INFO:tasks.workunit.client.0.smithi132.stdout: misc: 2024-03-13T08:06:58.980 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [----------] Global test environment tear-down 2024-03-13T08:06:58.980 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [==========] 12 tests from 1 test suite ran. (1866813 ms total) 2024-03-13T08:06:58.980 INFO:tasks.workunit.client.0.smithi132.stdout: misc: [ PASSED ] 12 tests. 2024-03-13T08:07:03.991 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:27.881 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:04.054 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:27.943 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:04.056 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:27.927 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:04.061 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:27.931 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:04.094 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:27.964 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:04.113 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:28.003 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:04.124 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:28.014 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:04.133 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:28.021 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:04.159 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:28.049 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:14.097 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:37.987 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:14.102 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:37.992 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:14.141 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:38.031 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:14.148 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:38.018 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:14.153 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:38.024 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:14.161 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:38.051 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:14.171 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:38.061 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:14.179 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:38.049 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:14.196 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:38.086 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:19.029 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:42.919 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:19.037 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:42.927 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:19.053 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:42.943 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:19.086 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:42.976 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:19.088 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:42.978 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:19.101 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:42.971 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:19.106 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:42.976 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:19.107 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:42.997 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:19.125 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:42.995 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:34.226 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:58.115 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:34.228 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:58.118 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:34.228 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:58.098 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:34.232 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:58.102 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:34.272 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:33:58.142 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:34.275 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:33:58.165 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:34.287 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:58.176 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:34.293 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:58.183 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:34.342 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:33:58.232 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:48.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-03-13T08:07:48.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-03-13T08:07:48.829 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-03-13T08:07:48.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-03-13T08:07:48.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-03-13T08:07:48.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (6259 ms) 2024-03-13T08:07:48.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-03-13T08:07:48.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (1823312 ms) 2024-03-13T08:07:48.830 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-03-13T08:07:48.831 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6734 ms) 2024-03-13T08:07:48.831 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-03-13T08:07:48.831 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (3139 ms) 2024-03-13T08:07:48.831 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-03-13T08:07:48.831 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3842 ms) 2024-03-13T08:07:48.831 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-03-13T08:07:48.832 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (3331 ms) 2024-03-13T08:07:48.832 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-03-13T08:07:48.832 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (3396 ms) 2024-03-13T08:07:48.832 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-03-13T08:07:48.832 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3436 ms) 2024-03-13T08:07:48.833 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-03-13T08:07:48.833 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (3495 ms) 2024-03-13T08:07:48.833 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-03-13T08:07:48.833 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3425 ms) 2024-03-13T08:07:48.833 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-03-13T08:07:48.833 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3557 ms) 2024-03-13T08:07:48.834 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-03-13T08:07:48.834 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (4396 ms) 2024-03-13T08:07:48.834 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-03-13T08:07:48.834 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3054 ms) 2024-03-13T08:07:48.834 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-03-13T08:07:48.835 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3322 ms) 2024-03-13T08:07:48.835 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-03-13T08:07:48.835 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3375 ms) 2024-03-13T08:07:48.835 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-03-13T08:07:48.835 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3357 ms) 2024-03-13T08:07:48.836 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-03-13T08:07:48.836 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3315 ms) 2024-03-13T08:07:48.836 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-03-13T08:07:48.836 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3332 ms) 2024-03-13T08:07:48.836 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-03-13T08:07:48.836 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3357 ms) 2024-03-13T08:07:48.836 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-03-13T08:07:48.837 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3022 ms) 2024-03-13T08:07:48.837 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-03-13T08:07:48.837 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3323 ms) 2024-03-13T08:07:48.837 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-03-13T08:07:48.837 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3271 ms) 2024-03-13T08:07:48.837 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-03-13T08:07:48.837 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (2569 ms) 2024-03-13T08:07:48.838 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-03-13T08:07:48.838 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3338 ms) 2024-03-13T08:07:48.838 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-03-13T08:07:48.838 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3437 ms) 2024-03-13T08:07:48.838 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-03-13T08:07:48.838 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3341 ms) 2024-03-13T08:07:48.839 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-03-13T08:07:49.339 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:13.228 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:49.346 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:13.236 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:49.379 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:13.269 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:49.417 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:13.288 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:49.419 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:13.309 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:49.423 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:13.293 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:49.425 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:13.315 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:49.459 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:13.329 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:49.463 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:13.353 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:54.106 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:17.994 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:54.148 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:18.018 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:54.149 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:18.020 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:54.150 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:18.040 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:54.153 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:18.044 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:54.154 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:18.044 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:54.164 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:18.034 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:54.169 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:18.059 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:07:54.170 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:18.060 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:04.208 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:28.098 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:04.213 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:28.103 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:04.239 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:28.129 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:04.258 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:28.128 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:04.259 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:28.130 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:04.268 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:28.158 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:04.270 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:28.160 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:04.280 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:28.150 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:04.304 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:28.194 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:14.241 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:38.131 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:14.242 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:38.133 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:14.266 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:38.156 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:14.289 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:38.159 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:14.290 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:38.161 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:14.291 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:38.181 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:14.293 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:38.183 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:14.312 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:38.183 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:14.315 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:38.205 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:19.277 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:43.167 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:19.280 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:43.170 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:19.280 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:43.151 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:19.294 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:43.183 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:19.301 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:43.171 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:19.314 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:43.184 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:19.325 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:43.215 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:19.328 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:43.218 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:19.341 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:43.231 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:34.525 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:58.414 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:34.528 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:58.418 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:34.572 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:34:58.462 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:34.577 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:58.447 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:34.578 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:58.448 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:34.587 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:58.477 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:34.588 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:58.478 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:34.609 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:34:58.479 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:34.620 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:34:58.509 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:49.435 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:35:13.324 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:49.484 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:35:13.374 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:49.514 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:35:13.404 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:49.530 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:35:13.400 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:49.538 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:35:13.409 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:49.543 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:35:13.433 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:49.547 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:35:13.437 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:49.566 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:35:13.436 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:08:49.576 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:35:13.466 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:00.666 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:35:24.556 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:00.669 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:35:24.559 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:00.696 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:35:24.586 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:00.720 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:35:24.609 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:00.720 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:35:24.591 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:00.721 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:35:24.592 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:00.723 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:35:24.613 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:00.747 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:35:24.617 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:00.750 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:35:24.640 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:04.635 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:35:28.525 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:04.639 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:35:28.529 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:04.652 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:35:28.542 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:04.684 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:35:28.574 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:04.686 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:35:28.576 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:04.690 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:35:28.561 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:04.692 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:35:28.562 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:04.699 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:35:28.589 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:04.708 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:35:28.578 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:19.803 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:35:43.693 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:19.807 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:35:43.697 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:19.850 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:35:43.740 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:19.878 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:35:43.748 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:19.890 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:35:43.780 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:19.900 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:35:43.770 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:19.910 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:35:43.800 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:19.925 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:35:43.795 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:19.938 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:35:43.828 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:39.745 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:03.635 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:39.793 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:03.683 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:39.831 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:03.721 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:39.842 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:03.732 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:39.843 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:03.733 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:39.846 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:03.716 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:39.853 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:03.721 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:39.887 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:03.757 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:39.889 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:03.779 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:49.829 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:13.718 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:49.832 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:13.722 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:49.854 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:13.744 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:49.884 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:13.754 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:49.894 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:13.765 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:49.899 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:13.789 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:49.905 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:13.794 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:49.917 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:13.787 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:49.929 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:13.819 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:59.828 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:23.717 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:59.830 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:23.720 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:59.853 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:23.743 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:59.881 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:23.770 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:59.882 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:23.772 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:59.903 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:23.793 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:59.912 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:23.782 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:59.923 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:23.793 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:09:59.942 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:23.812 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:04.759 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:28.649 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:04.762 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:28.652 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:04.782 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:28.672 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:04.808 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:28.678 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:04.809 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:28.679 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:04.813 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:28.703 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:04.817 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:28.707 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:04.823 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:28.694 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:04.831 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:28.720 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:19.993 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:43.882 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:20.001 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:43.889 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:20.027 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:43.917 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:20.051 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:43.940 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:20.052 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:43.942 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:20.072 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:43.942 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:20.079 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:43.949 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:20.089 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:43.979 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:20.118 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:43.988 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:21.037 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripAppendPP (3419 ms) 2024-03-13T08:10:21.038 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-03-13T08:10:21.038 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (2388 ms) 2024-03-13T08:10:21.038 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-03-13T08:10:21.038 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3595 ms) 2024-03-13T08:10:21.038 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-03-13T08:10:21.038 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3219 ms) 2024-03-13T08:10:21.038 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-03-13T08:10:21.038 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: setting pool EIO 2024-03-13T08:10:21.038 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: max_success 0, min_failed 1 2024-03-13T08:10:21.038 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (5832 ms) 2024-03-13T08:10:21.038 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-03-13T08:10:21.038 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3137 ms) 2024-03-13T08:10:21.039 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (1935325 ms total) 2024-03-13T08:10:21.039 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: 2024-03-13T08:10:21.039 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-03-13T08:10:21.039 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-03-13T08:10:21.039 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3557 ms) 2024-03-13T08:10:21.039 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-03-13T08:10:21.039 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (10361 ms) 2024-03-13T08:10:21.039 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-03-13T08:10:21.039 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (16914 ms) 2024-03-13T08:10:21.039 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-03-13T08:10:21.039 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (2776 ms) 2024-03-13T08:10:21.039 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (33608 ms total) 2024-03-13T08:10:21.039 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: 2024-03-13T08:10:21.039 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-03-13T08:10:21.040 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-03-13T08:10:21.040 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3431 ms) 2024-03-13T08:10:21.040 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3431 ms total) 2024-03-13T08:10:21.040 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: 2024-03-13T08:10:21.040 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-03-13T08:10:21.040 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-03-13T08:10:21.040 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (14866 ms) 2024-03-13T08:10:21.040 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-03-13T08:10:21.040 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6680 ms) 2024-03-13T08:10:21.040 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-03-13T08:10:21.040 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7893 ms) 2024-03-13T08:10:21.040 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-03-13T08:10:21.040 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7578 ms) 2024-03-13T08:10:21.040 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-03-13T08:10:21.040 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3090 ms) 2024-03-13T08:10:21.041 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-03-13T08:10:21.041 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (6620 ms) 2024-03-13T08:10:21.041 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-03-13T08:10:21.041 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (7746 ms) 2024-03-13T08:10:21.041 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-03-13T08:10:21.041 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7637 ms) 2024-03-13T08:10:21.041 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-03-13T08:10:21.041 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (6771 ms) 2024-03-13T08:10:21.041 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-03-13T08:10:21.041 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (6850 ms) 2024-03-13T08:10:21.041 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-03-13T08:10:21.041 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7663 ms) 2024-03-13T08:10:21.041 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-03-13T08:10:21.041 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (6926 ms) 2024-03-13T08:10:21.042 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-03-13T08:10:21.042 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (6679 ms) 2024-03-13T08:10:34.851 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:58.741 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:34.897 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:58.767 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:34.898 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:58.768 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:34.900 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:58.790 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:34.927 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:36:58.797 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:34.945 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:36:58.834 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:34.965 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:58.854 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:34.967 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:58.856 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:34.997 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:36:58.884 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:44.976 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:08.866 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:45.024 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:08.894 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:45.025 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:08.915 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:45.028 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:08.898 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:45.049 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:08.919 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:45.064 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:08.953 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:45.090 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:08.980 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:45.096 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:08.985 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:45.119 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:09.009 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:49.893 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:13.783 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:49.939 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:13.809 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:49.940 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:13.830 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:49.942 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:13.813 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:49.956 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:13.826 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:49.966 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:13.855 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:49.989 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:13.879 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:49.995 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:13.883 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:50.008 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:13.898 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:53.892 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:17.762 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:53.926 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:17.815 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:53.935 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:17.825 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:53.940 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:17.810 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:53.945 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:17.835 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:53.958 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:17.848 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:53.963 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:17.833 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:54.005 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:17.895 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:10:54.030 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:17.919 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:05.114 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:29.003 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:05.174 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:29.043 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:05.175 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:29.045 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:05.184 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:29.072 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:05.209 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:29.079 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:05.227 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:29.117 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:05.244 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:29.133 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:05.249 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:29.139 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:05.285 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:29.174 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:08.240 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-03-13T08:11:08.240 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3343 ms) 2024-03-13T08:11:08.240 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-03-13T08:11:08.240 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (6547 ms) 2024-03-13T08:11:08.240 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-03-13T08:11:08.240 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (8000 ms) 2024-03-13T08:11:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-03-13T08:11:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7043 ms) 2024-03-13T08:11:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-03-13T08:11:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7988 ms) 2024-03-13T08:11:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-03-13T08:11:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7546 ms) 2024-03-13T08:11:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-03-13T08:11:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (6732 ms) 2024-03-13T08:11:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (144198 ms total) 2024-03-13T08:11:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: 2024-03-13T08:11:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-03-13T08:11:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (2116562 ms total) 2024-03-13T08:11:08.241 INFO:tasks.workunit.client.0.smithi132.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-03-13T08:11:08.244 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.244 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40788 2024-03-13T08:11:08.244 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 40788 2024-03-13T08:11:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41832 2024-03-13T08:11:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41832 2024-03-13T08:11:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41034 2024-03-13T08:11:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41034 2024-03-13T08:11:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41890 2024-03-13T08:11:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41890 2024-03-13T08:11:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41133 2024-03-13T08:11:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41133 2024-03-13T08:11:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41619 2024-03-13T08:11:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41619 2024-03-13T08:11:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41931 2024-03-13T08:11:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41931 2024-03-13T08:11:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41054 2024-03-13T08:11:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41054 2024-03-13T08:11:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41446 2024-03-13T08:11:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41446 2024-03-13T08:11:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41790 2024-03-13T08:11:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41790 2024-03-13T08:11:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41582 2024-03-13T08:11:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41582 2024-03-13T08:11:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41868 2024-03-13T08:11:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41868 2024-03-13T08:11:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41366 2024-03-13T08:11:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41366 2024-03-13T08:11:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40938 2024-03-13T08:11:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 40938 2024-03-13T08:11:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41916 2024-03-13T08:11:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41916 2024-03-13T08:11:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41272 2024-03-13T08:11:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41272 2024-03-13T08:11:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40760 2024-03-13T08:11:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 40760 2024-03-13T08:11:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40826 2024-03-13T08:11:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 40826 2024-03-13T08:11:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41228 2024-03-13T08:11:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41228 2024-03-13T08:11:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=40870 2024-03-13T08:11:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 40870 2024-03-13T08:11:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41304 2024-03-13T08:11:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41304 2024-03-13T08:11:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41323 2024-03-13T08:11:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41323 2024-03-13T08:11:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41770 2024-03-13T08:11:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41770 2024-03-13T08:11:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:+ for t in "${!pids[@]}" 2024-03-13T08:11:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:+ pid=41947 2024-03-13T08:11:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:+ wait 41947 2024-03-13T08:11:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:+ exit 0 2024-03-13T08:11:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:+ cleanup 2024-03-13T08:11:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:+ pkill -P 40754 2024-03-13T08:11:08.257 INFO:tasks.workunit.client.0.smithi132.stderr:+ true 2024-03-13T08:11:08.257 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-13T08:11:08.257 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-13T08:11:08.286 INFO:tasks.workunit:Stopping ['rados/test.sh'] on client.0... 2024-03-13T08:11:08.287 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-13T08:11:08.648 DEBUG:teuthology.parallel:result is None 2024-03-13T08:11:08.648 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-13T08:11:08.676 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-13T08:11:08.676 DEBUG:teuthology.orchestra.run.smithi132:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-13T08:11:08.736 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-13T08:11:08.736 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-13T08:11:08.746 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-13T08:11:08.747 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-13T08:11:08.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.879+0000 7f2a44675640 10 monclient: get_monmap_and_config 2024-03-13T08:11:08.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.879+0000 7f2a44675640 10 monclient: build_initial_monmap 2024-03-13T08:11:08.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.879+0000 7f2a44675640 10 monclient: monmap: 2024-03-13T08:11:08.880 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:08.880 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:08.880 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:08.879969+0000 2024-03-13T08:11:08.880 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:08.879969+0000 2024-03-13T08:11:08.880 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:08.880 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:08.880 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:08.880 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:08.880 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:08.880 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:08.880 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:08.880 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:08.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.880+0000 7f2a44675640 1 Processor -- start 2024-03-13T08:11:08.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.880+0000 7f2a44675640 1 -- start start 2024-03-13T08:11:08.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.880+0000 7f2a44675640 10 monclient: init 2024-03-13T08:11:08.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.880+0000 7f2a44675640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:08.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.880+0000 7f2a44675640 10 monclient: _add_conns ranks=[1,3,2,0,4] 2024-03-13T08:11:08.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.880+0000 7f2a44675640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2a3c0b1e70 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:08.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.880+0000 7f2a44675640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2a3c0b00f0 0x7f2a3c13fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:08.884 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.880+0000 7f2a44675640 10 monclient(hunting): picked mon.noname-a con 0x7f2a3c0b00f0 addr v2:172.21.15.132:3300/0 2024-03-13T08:11:08.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.880+0000 7f2a44675640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a3c140400 0x7f2a3c142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:08.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.880+0000 7f2a44675640 10 monclient(hunting): picked mon.noname-b con 0x7f2a3c140400 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:08.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.881+0000 7f2a44675640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a3c0b1390 con 0x7f2a3c0b1e70 2024-03-13T08:11:08.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.881+0000 7f2a44675640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:08.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.881+0000 7f2a44675640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c0b1670 con 0x7f2a3c0b00f0 2024-03-13T08:11:08.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.881+0000 7f2a44675640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:08.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.881+0000 7f2a44675640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0b05a0 con 0x7f2a3c140400 2024-03-13T08:11:08.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.881+0000 7f2a44675640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:08.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.881+0000 7f2a44675640 10 monclient(hunting): authenticate will time out at 3496.289125s 2024-03-13T08:11:08.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.881+0000 7f2a41c10640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2a3c0b00f0 0x7f2a3c13fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:08.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.881+0000 7f2a42411640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f2a3c0b1e70 0x7f2a3c0b2250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.132:41574/0 (socket says 172.21.15.132:41574) 2024-03-13T08:11:08.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.881+0000 7f2a42411640 1 -- 172.21.15.132:0/1814015641 learned_addr learned my addr 172.21.15.132:0/1814015641 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T08:11:08.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.882+0000 7f2a41c10640 10 monclient(hunting): get_auth_request con 0x7f2a3c0b00f0 auth_method 0 2024-03-13T08:11:08.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.882+0000 7f2a41c10640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:08.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.882+0000 7f2a41c10640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:08.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.882+0000 7f2a41c10640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:08.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.882+0000 7f2a41c10640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:08.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.882+0000 7f2a41c10640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:08.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.882+0000 7f2a41c10640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:08.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.883+0000 7f2a42c12640 1 --2- 172.21.15.132:0/1814015641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a3c140400 0x7f2a3c142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:08.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.884+0000 7f2a42c12640 10 monclient(hunting): get_auth_request con 0x7f2a3c140400 auth_method 0 2024-03-13T08:11:08.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.884+0000 7f2a42c12640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:08.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.884+0000 7f2a42c12640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:08.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.884+0000 7f2a42c12640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:08.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.884+0000 7f2a41c10640 10 monclient(hunting): handle_auth_done global_id 15996 payload 274 2024-03-13T08:11:08.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.884+0000 7f2a4140f640 1 -- 172.21.15.132:0/1814015641 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271111797 0 0) 0x7f2a3c0b1390 con 0x7f2a3c0b1e70 2024-03-13T08:11:08.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.884+0000 7f2a41c10640 1 -- 172.21.15.132:0/1814015641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a3c140400 msgr2=0x7f2a3c142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:08.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.884+0000 7f2a41c10640 1 --2- 172.21.15.132:0/1814015641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a3c140400 0x7f2a3c142ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:08.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.884+0000 7f2a41c10640 1 -- 172.21.15.132:0/1814015641 >> v1:172.21.15.195:6789/0 conn(0x7f2a3c0b1e70 legacy=0x7f2a3c0b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:08.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.884+0000 7f2a41c10640 10 monclient: _finish_hunting 0 2024-03-13T08:11:08.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.884+0000 7f2a41c10640 1 monclient: found mon.noname-a 2024-03-13T08:11:08.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.884+0000 7f2a41c10640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:08.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.884+0000 7f2a41c10640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T08:11:08.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.884+0000 7f2a41c10640 1 -- 172.21.15.132:0/1814015641 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c143010 con 0x7f2a3c0b00f0 2024-03-13T08:11:08.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.884+0000 7f2a4140f640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-03-13T08:11:08.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.886+0000 7f2a41c10640 1 --2- 172.21.15.132:0/1814015641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a3c0b00f0 0x7f2a3c13fec0 secure :-1 s=READY pgs=3258 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0097a0 tx=0x7f2a2c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d51bb548ca4626 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:08.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a4140f640 1 -- 172.21.15.132:0/1814015641 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c035020 con 0x7f2a3c0b00f0 2024-03-13T08:11:08.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a4140f640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:08.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a4140f640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T08:11:08.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a4140f640 10 monclient: dump: 2024-03-13T08:11:08.895 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:08.895 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:08.896 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:08.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a4140f640 10 monclient: _finish_auth 0 2024-03-13T08:11:08.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a4140f640 10 monclient: _check_auth_tickets 2024-03-13T08:11:08.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a4140f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:08.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a4140f640 1 -- 172.21.15.132:0/1814015641 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2c031830 con 0x7f2a3c0b00f0 2024-03-13T08:11:08.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a44675640 5 monclient: authenticate success, global_id 15996 2024-03-13T08:11:08.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a44675640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T08:11:08.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a4140f640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:08.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a44675640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:08.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a44675640 4 set_mon_vals no callback set 2024-03-13T08:11:08.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a4140f640 1 -- 172.21.15.132:0/1814015641 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c031c00 con 0x7f2a3c0b00f0 2024-03-13T08:11:08.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a4140f640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:08.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a4140f640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T08:11:08.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.890+0000 7f2a4140f640 10 monclient: dump: 2024-03-13T08:11:08.897 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:08.897 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:08.898 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:08.898 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:08.898 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:08.898 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:08.898 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:08.898 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:08.898 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:08.898 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:08.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.891+0000 7f2a44675640 10 monclient: shutdown 2024-03-13T08:11:08.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.891+0000 7f2a44675640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:08.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.891+0000 7f2a44675640 1 -- 172.21.15.132:0/1814015641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a3c0b00f0 msgr2=0x7f2a3c13fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:08.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.891+0000 7f2a44675640 1 --2- 172.21.15.132:0/1814015641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a3c0b00f0 0x7f2a3c13fec0 secure :-1 s=READY pgs=3258 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0097a0 tx=0x7f2a2c02b8d0 comp rx=0 tx=0).stop 2024-03-13T08:11:08.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.891+0000 7f2a44675640 1 -- 172.21.15.132:0/1814015641 shutdown_connections 2024-03-13T08:11:08.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.891+0000 7f2a44675640 1 --2- 172.21.15.132:0/1814015641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a3c140400 0x7f2a3c142ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:08.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.891+0000 7f2a44675640 1 --2- 172.21.15.132:0/1814015641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a3c0b00f0 0x7f2a3c13fec0 unknown :-1 s=CLOSED pgs=3258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:08.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.891+0000 7f2a44675640 1 -- 172.21.15.132:0/1814015641 >> 172.21.15.132:0/1814015641 conn(0x7f2a3c0b80e0 msgr2=0x7f2a3c0b84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:08.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.891+0000 7f2a44675640 1 -- 172.21.15.132:0/1814015641 shutdown_connections 2024-03-13T08:11:08.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.891+0000 7f2a44675640 1 -- 172.21.15.132:0/1814015641 wait complete. 2024-03-13T08:11:08.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.892+0000 7f2a44675640 10 monclient: build_initial_monmap 2024-03-13T08:11:08.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.892+0000 7f2a44675640 10 monclient: monmap: 2024-03-13T08:11:08.899 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:08.899 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:08.900 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:08.900 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:08.900 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:08.900 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:08.900 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:08.900 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:08.900 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:08.901 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:08.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.892+0000 7f2a44675640 1 librados: starting msgr at 2024-03-13T08:11:08.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.892+0000 7f2a44675640 1 librados: starting objecter 2024-03-13T08:11:08.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.892+0000 7f2a44675640 1 Processor -- start 2024-03-13T08:11:08.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.892+0000 7f2a44675640 1 -- start start 2024-03-13T08:11:08.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.892+0000 7f2a44675640 1 librados: setting wanted keys 2024-03-13T08:11:08.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.892+0000 7f2a44675640 1 librados: calling monclient init 2024-03-13T08:11:08.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.892+0000 7f2a44675640 10 monclient: init 2024-03-13T08:11:08.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a44675640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:08.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a44675640 10 monclient: _add_conns ranks=[1,0,2] 2024-03-13T08:11:08.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a44675640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a3c0b00f0 0x7f2a3c1db460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:08.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a44675640 10 monclient(hunting): picked mon.noname-b con 0x7f2a3c0b00f0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:08.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a44675640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a3c0b1e70 0x7f2a3c0ab100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:08.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a44675640 10 monclient(hunting): picked mon.noname-a con 0x7f2a3c0b1e70 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:08.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a44675640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a3c140400 0x7f2a3c1d78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:08.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a44675640 10 monclient(hunting): picked mon.noname-c con 0x7f2a3c140400 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:08.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a44675640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:08.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a44675640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c143670 con 0x7f2a3c0b1e70 2024-03-13T08:11:08.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a44675640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:08.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a44675640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c14d290 con 0x7f2a3c0b00f0 2024-03-13T08:11:08.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a44675640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:08.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a44675640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c1dafd0 con 0x7f2a3c140400 2024-03-13T08:11:08.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a44675640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:08.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a44675640 10 monclient(hunting): authenticate will time out at 3496.301348s 2024-03-13T08:11:08.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a41c10640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a3c0b1e70 0x7f2a3c0ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:08.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a42411640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a3c0b00f0 0x7f2a3c1db460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:08.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.893+0000 7f2a42c12640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a3c140400 0x7f2a3c1d78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:08.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a41c10640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a3c0b1e70 0x7f2a3c0ab100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51052/0 (socket says 172.21.15.132:51052) 2024-03-13T08:11:08.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a41c10640 1 -- 172.21.15.132:0/3181350802 learned_addr learned my addr 172.21.15.132:0/3181350802 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:08.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a41c10640 10 monclient(hunting): get_auth_request con 0x7f2a3c0b1e70 auth_method 0 2024-03-13T08:11:08.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a41c10640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:08.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a41c10640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:08.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a41c10640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:08.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a42411640 10 monclient(hunting): get_auth_request con 0x7f2a3c0b00f0 auth_method 0 2024-03-13T08:11:08.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a42411640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:08.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a42411640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:08.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a42411640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:08.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a42c12640 10 monclient(hunting): get_auth_request con 0x7f2a3c140400 auth_method 0 2024-03-13T08:11:08.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a42c12640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:08.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a42c12640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:08.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a42c12640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:08.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a42411640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:08.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a42411640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:08.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a42411640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:08.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a41c10640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:08.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a41c10640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:08.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.894+0000 7f2a41c10640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:08.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.895+0000 7f2a42c12640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:08.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.895+0000 7f2a42c12640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:08.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.895+0000 7f2a42c12640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:08.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.897+0000 7f2a42411640 10 monclient(hunting): handle_auth_done global_id 19645 payload 824 2024-03-13T08:11:08.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.897+0000 7f2a42411640 1 -- 172.21.15.132:0/3181350802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a3c140400 msgr2=0x7f2a3c1d78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:08.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.897+0000 7f2a42411640 1 --2- 172.21.15.132:0/3181350802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a3c140400 0x7f2a3c1d78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:08.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.897+0000 7f2a42411640 1 -- 172.21.15.132:0/3181350802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a3c0b1e70 msgr2=0x7f2a3c0ab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:08.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.897+0000 7f2a42411640 1 --2- 172.21.15.132:0/3181350802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a3c0b1e70 0x7f2a3c0ab100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:08.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.897+0000 7f2a42411640 10 monclient: _finish_hunting 0 2024-03-13T08:11:08.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.897+0000 7f2a42411640 1 monclient: found mon.noname-b 2024-03-13T08:11:08.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.897+0000 7f2a42411640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:08.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.897+0000 7f2a42411640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T08:11:08.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.897+0000 7f2a42411640 1 -- 172.21.15.132:0/3181350802 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c0ac8e0 con 0x7f2a3c0b00f0 2024-03-13T08:11:08.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.899+0000 7f2a42411640 1 --2- 172.21.15.132:0/3181350802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a3c0b00f0 0x7f2a3c1db460 secure :-1 s=READY pgs=4722 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800c450 tx=0x7f2a3800c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:08.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.899+0000 7f2a32ffd640 10 client.?.objecter ms_handle_connect 0x7f2a3c0b00f0 2024-03-13T08:11:08.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.899+0000 7f2a32ffd640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:08.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a32ffd640 1 -- 172.21.15.132:0/3181350802 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38018070 con 0x7f2a3c0b00f0 2024-03-13T08:11:08.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a32ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:08.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a32ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:08.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a32ffd640 10 monclient: dump: 2024-03-13T08:11:08.907 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:08.907 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:08.907 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:08.907 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:08.907 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:08.907 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:08.907 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:08.907 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:08.907 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:08.907 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:08.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a32ffd640 10 monclient: _finish_auth 0 2024-03-13T08:11:08.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a32ffd640 10 monclient: _check_auth_tickets 2024-03-13T08:11:08.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a32ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:08.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a32ffd640 1 -- 172.21.15.132:0/3181350802 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a380074e0 con 0x7f2a3c0b00f0 2024-03-13T08:11:08.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a44675640 5 monclient: authenticate success, global_id 19645 2024-03-13T08:11:08.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a44675640 10 monclient: _renew_subs 2024-03-13T08:11:08.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a44675640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:08.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a44675640 1 -- 172.21.15.132:0/3181350802 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3c0aa550 con 0x7f2a3c0b00f0 2024-03-13T08:11:08.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a32ffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:08.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a32ffd640 1 -- 172.21.15.132:0/3181350802 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38004a20 con 0x7f2a3c0b00f0 2024-03-13T08:11:08.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a32ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:08.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a32ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:08.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a32ffd640 10 monclient: dump: 2024-03-13T08:11:08.908 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:08.908 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:08.908 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:08.908 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:08.909 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:08.909 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:08.909 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:08.909 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:08.909 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:08.909 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:08.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a40c0e640 4 set_mon_vals no callback set 2024-03-13T08:11:08.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a44675640 10 client.19645.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:08.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a44675640 10 monclient: _renew_subs 2024-03-13T08:11:08.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a44675640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:08.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a44675640 1 -- 172.21.15.132:0/3181350802 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c0ad210 con 0x7f2a3c0b00f0 2024-03-13T08:11:08.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.903+0000 7f2a44675640 1 librados: init done 2024-03-13T08:11:08.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.904+0000 7f2a44675640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:08.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.904+0000 7f2a44675640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:08.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.904+0000 7f2a44675640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:08.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.904+0000 7f2a44675640 1 -- 172.21.15.132:0/3181350802 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c04f000 con 0x7f2a3c0b00f0 2024-03-13T08:11:08.926 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.925+0000 7f2a32ffd640 1 -- 172.21.15.132:0/3181350802 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f2a3801b070 con 0x7f2a3c0b00f0 2024-03-13T08:11:08.926 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.925+0000 7f2a32ffd640 1 --2- 172.21.15.132:0/3181350802 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2a2006d560 0x7f2a2006fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:08.926 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.925+0000 7f2a41c10640 1 --2- 172.21.15.132:0/3181350802 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2a2006d560 0x7f2a2006fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:08.926 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.925+0000 7f2a41c10640 10 monclient: get_auth_request con 0x7f2a2006d560 auth_method 0 2024-03-13T08:11:08.926 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.926+0000 7f2a32ffd640 10 client.19645.objecter ms_handle_connect 0x7f2a2006d560 2024-03-13T08:11:08.926 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.926+0000 7f2a41c10640 1 --2- 172.21.15.132:0/3181350802 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2a2006d560 0x7f2a2006fa20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0a9590 tx=0x7f2a2c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:08.928 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.927+0000 7f2a32ffd640 1 -- 172.21.15.132:0/3181350802 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1292..1292 src has 1008..1292) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f2a38056530 con 0x7f2a3c0b00f0 2024-03-13T08:11:08.928 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.927+0000 7f2a32ffd640 10 client.19645.objecter ms_dispatch 0x7f2a3c000b90 osd_map(1292..1292 src has 1008..1292) v4 2024-03-13T08:11:08.928 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.927+0000 7f2a32ffd640 3 client.19645.objecter handle_osd_map got epochs [1292,1292] > 0 2024-03-13T08:11:08.928 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.927+0000 7f2a32ffd640 3 client.19645.objecter handle_osd_map decoding full epoch 1292 2024-03-13T08:11:08.928 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:08.927+0000 7f2a32ffd640 20 client.19645.objecter dump_active .. 0 homeless 2024-03-13T08:11:09.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.159+0000 7f2a32ffd640 1 -- 172.21.15.132:0/3181350802 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f2a38058490 con 0x7f2a3c0b00f0 2024-03-13T08:11:09.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.159+0000 7f2a32ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:09.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.159+0000 7f2a32ffd640 10 monclient: _finish_command 1 = system:0 2024-03-13T08:11:09.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.338+0000 7f2a44675640 1 -- 172.21.15.132:0/3181350802 --> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a3c067e10 con 0x7f2a2006d560 2024-03-13T08:11:09.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.342+0000 7f2a32ffd640 1 -- 172.21.15.132:0/3181350802 <== mgr.4104 v2:172.21.15.132:6800/3847550149 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+116241 (secure 0 0 0) 0x7f2a3c067e10 con 0x7f2a2006d560 2024-03-13T08:11:09.343 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T08:11:09.344 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-03-13T08:11:09.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.344+0000 7f2a44675640 10 librados: watch_flush enter 2024-03-13T08:11:09.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.344+0000 7f2a44675640 10 librados: watch_flush exit 2024-03-13T08:11:09.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.344+0000 7f2a44675640 20 client.19645.objecter shutdown clearing up homeless session... 2024-03-13T08:11:09.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.344+0000 7f2a44675640 10 client.19645.objecter successfully canceled tick 2024-03-13T08:11:09.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.344+0000 7f2a44675640 1 -- 172.21.15.132:0/3181350802 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2a2006d560 msgr2=0x7f2a2006fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:09.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.344+0000 7f2a44675640 1 --2- 172.21.15.132:0/3181350802 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2a2006d560 0x7f2a2006fa20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0a9590 tx=0x7f2a2c008030 comp rx=0 tx=0).stop 2024-03-13T08:11:09.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.344+0000 7f2a44675640 10 monclient: shutdown 2024-03-13T08:11:09.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.344+0000 7f2a44675640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:09.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.344+0000 7f2a44675640 1 -- 172.21.15.132:0/3181350802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a3c0b00f0 msgr2=0x7f2a3c1db460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:09.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.344+0000 7f2a44675640 1 --2- 172.21.15.132:0/3181350802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a3c0b00f0 0x7f2a3c1db460 secure :-1 s=READY pgs=4722 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800c450 tx=0x7f2a3800c910 comp rx=0 tx=0).stop 2024-03-13T08:11:09.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.345+0000 7f2a44675640 1 -- 172.21.15.132:0/3181350802 shutdown_connections 2024-03-13T08:11:09.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.345+0000 7f2a44675640 1 --2- 172.21.15.132:0/3181350802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a3c140400 0x7f2a3c1d78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:09.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.345+0000 7f2a44675640 1 --2- 172.21.15.132:0/3181350802 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f2a2006d560 0x7f2a2006fa20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:09.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.345+0000 7f2a44675640 1 --2- 172.21.15.132:0/3181350802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a3c0b1e70 0x7f2a3c0ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:09.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.345+0000 7f2a44675640 1 --2- 172.21.15.132:0/3181350802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a3c0b00f0 0x7f2a3c1db460 unknown :-1 s=CLOSED pgs=4722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:09.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.345+0000 7f2a44675640 1 -- 172.21.15.132:0/3181350802 >> 172.21.15.132:0/3181350802 conn(0x7f2a3c0b80e0 msgr2=0x7f2a3c0b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:09.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.345+0000 7f2a44675640 1 -- 172.21.15.132:0/3181350802 shutdown_connections 2024-03-13T08:11:09.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.345+0000 7f2a44675640 1 -- 172.21.15.132:0/3181350802 wait complete. 2024-03-13T08:11:09.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.345+0000 7f2a44675640 1 librados: shutdown 2024-03-13T08:11:09.356 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":2393,"stamp":"2024-03-13T08:11:08.179017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":577148,"kb_used_data":38900,"kb_used_omap":12,"kb_used_meta":538099,"kb_avail":754397572,"statfs":{"total":773094113280,"available":772503113728,"internally_reserved":0,"allocated":39833600,"data_stored":18490102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":551013938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":160,"apply_latency_ms":160,"commit_latency_ns":160000000,"apply_latency_ns":160000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.534546"},"pg_stats":[{"pgid":"9.19","version":"0'0","reported_seq":2485,"reported_epoch":1286,"state":"active+clean","last_fresh":"2024-03-13T08:11:02.487636+0000","last_change":"2024-03-13T07:35:58.099771+0000","last_active":"2024-03-13T08:11:02.487636+0000","last_peered":"2024-03-13T08:11:02.487636+0000","last_clean":"2024-03-13T08:11:02.487636+0000","last_became_active":"2024-03-13T07:35:58.062655+0000","last_became_peered":"2024-03-13T07:35:58.062655+0000","last_unstale":"2024-03-13T08:11:02.487636+0000","last_undegraded":"2024-03-13T08:11:02.487636+0000","last_fullsized":"2024-03-13T08:11:02.487636+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:18:10.709550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":2481,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.069963+0000","last_change":"2024-03-13T07:35:56.417609+0000","last_active":"2024-03-13T08:11:05.069963+0000","last_peered":"2024-03-13T08:11:05.069963+0000","last_clean":"2024-03-13T08:11:05.069963+0000","last_became_active":"2024-03-13T07:35:56.409964+0000","last_became_peered":"2024-03-13T07:35:56.409964+0000","last_unstale":"2024-03-13T08:11:05.069963+0000","last_undegraded":"2024-03-13T08:11:05.069963+0000","last_fullsized":"2024-03-13T08:11:05.069963+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T07:57:49.564938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":2481,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.140120+0000","last_change":"2024-03-13T07:35:59.115829+0000","last_active":"2024-03-13T08:11:05.140120+0000","last_peered":"2024-03-13T08:11:05.140120+0000","last_clean":"2024-03-13T08:11:05.140120+0000","last_became_active":"2024-03-13T07:35:59.109130+0000","last_became_peered":"2024-03-13T07:35:59.109130+0000","last_unstale":"2024-03-13T08:11:05.140120+0000","last_undegraded":"2024-03-13T08:11:05.140120+0000","last_fullsized":"2024-03-13T08:11:05.140120+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:21:21.572511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":2485,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.267971+0000","last_change":"2024-03-13T07:36:26.745752+0000","last_active":"2024-03-13T08:11:07.267971+0000","last_peered":"2024-03-13T08:11:07.267971+0000","last_clean":"2024-03-13T08:11:07.267971+0000","last_became_active":"2024-03-13T07:36:26.737387+0000","last_became_peered":"2024-03-13T07:36:26.737387+0000","last_unstale":"2024-03-13T08:11:07.267971+0000","last_undegraded":"2024-03-13T08:11:07.267971+0000","last_fullsized":"2024-03-13T08:11:07.267971+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:06:10.706337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":2488,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.344331+0000","last_change":"2024-03-13T07:35:56.978987+0000","last_active":"2024-03-13T08:11:07.344331+0000","last_peered":"2024-03-13T08:11:07.344331+0000","last_clean":"2024-03-13T08:11:07.344331+0000","last_became_active":"2024-03-13T07:35:56.973580+0000","last_became_peered":"2024-03-13T07:35:56.973580+0000","last_unstale":"2024-03-13T08:11:07.344331+0000","last_undegraded":"2024-03-13T08:11:07.344331+0000","last_fullsized":"2024-03-13T08:11:07.344331+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:11:56.932620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":2473,"reported_epoch":1287,"state":"active+clean","last_fresh":"2024-03-13T08:11:04.025237+0000","last_change":"2024-03-13T07:35:56.488106+0000","last_active":"2024-03-13T08:11:04.025237+0000","last_peered":"2024-03-13T08:11:04.025237+0000","last_clean":"2024-03-13T08:11:04.025237+0000","last_became_active":"2024-03-13T07:35:56.482614+0000","last_became_peered":"2024-03-13T07:35:56.482614+0000","last_unstale":"2024-03-13T08:11:04.025237+0000","last_undegraded":"2024-03-13T08:11:04.025237+0000","last_fullsized":"2024-03-13T08:11:04.025237+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:43:18.546715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":2483,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.097675+0000","last_change":"2024-03-13T07:35:56.873267+0000","last_active":"2024-03-13T08:11:05.097675+0000","last_peered":"2024-03-13T08:11:05.097675+0000","last_clean":"2024-03-13T08:11:05.097675+0000","last_became_active":"2024-03-13T07:35:56.864351+0000","last_became_peered":"2024-03-13T07:35:56.864351+0000","last_unstale":"2024-03-13T08:11:05.097675+0000","last_undegraded":"2024-03-13T08:11:05.097675+0000","last_fullsized":"2024-03-13T08:11:05.097675+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T16:48:28.626092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":2483,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.085867+0000","last_change":"2024-03-13T07:35:55.887861+0000","last_active":"2024-03-13T08:11:05.085867+0000","last_peered":"2024-03-13T08:11:05.085867+0000","last_clean":"2024-03-13T08:11:05.085867+0000","last_became_active":"2024-03-13T07:35:55.874083+0000","last_became_peered":"2024-03-13T07:35:55.874083+0000","last_unstale":"2024-03-13T08:11:05.085867+0000","last_undegraded":"2024-03-13T08:11:05.085867+0000","last_fullsized":"2024-03-13T08:11:05.085867+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:55:42.390559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"44'7","reported_seq":2486,"reported_epoch":1287,"state":"active+clean","last_fresh":"2024-03-13T08:11:04.011026+0000","last_change":"2024-03-13T07:35:58.311435+0000","last_active":"2024-03-13T08:11:04.011026+0000","last_peered":"2024-03-13T08:11:04.011026+0000","last_clean":"2024-03-13T08:11:04.011026+0000","last_became_active":"2024-03-13T07:35:58.303418+0000","last_became_peered":"2024-03-13T07:35:58.303418+0000","last_unstale":"2024-03-13T08:11:04.011026+0000","last_undegraded":"2024-03-13T08:11:04.011026+0000","last_fullsized":"2024-03-13T08:11:04.011026+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:15:47.397228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":2487,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.240610+0000","last_change":"2024-03-13T07:36:00.388143+0000","last_active":"2024-03-13T08:11:07.240610+0000","last_peered":"2024-03-13T08:11:07.240610+0000","last_clean":"2024-03-13T08:11:07.240610+0000","last_became_active":"2024-03-13T07:36:00.379147+0000","last_became_peered":"2024-03-13T07:36:00.379147+0000","last_unstale":"2024-03-13T08:11:07.240610+0000","last_undegraded":"2024-03-13T08:11:07.240610+0000","last_fullsized":"2024-03-13T08:11:07.240610+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:55:32.801526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":2511,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.097762+0000","last_change":"2024-03-13T07:35:55.903931+0000","last_active":"2024-03-13T08:11:05.097762+0000","last_peered":"2024-03-13T08:11:05.097762+0000","last_clean":"2024-03-13T08:11:05.097762+0000","last_became_active":"2024-03-13T07:35:55.898189+0000","last_became_peered":"2024-03-13T07:35:55.898189+0000","last_unstale":"2024-03-13T08:11:05.097762+0000","last_undegraded":"2024-03-13T08:11:05.097762+0000","last_fullsized":"2024-03-13T08:11:05.097762+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T12:46:32.909808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":2487,"reported_epoch":1286,"state":"active+clean","last_fresh":"2024-03-13T08:11:02.503600+0000","last_change":"2024-03-13T07:35:59.285198+0000","last_active":"2024-03-13T08:11:02.503600+0000","last_peered":"2024-03-13T08:11:02.503600+0000","last_clean":"2024-03-13T08:11:02.503600+0000","last_became_active":"2024-03-13T07:35:59.276875+0000","last_became_peered":"2024-03-13T07:35:59.276875+0000","last_unstale":"2024-03-13T08:11:02.503600+0000","last_undegraded":"2024-03-13T08:11:02.503600+0000","last_fullsized":"2024-03-13T08:11:02.503600+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:08:00.548538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'76","reported_seq":2510,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.134477+0000","last_change":"2024-03-13T07:34:48.043475+0000","last_active":"2024-03-13T08:11:05.134477+0000","last_peered":"2024-03-13T08:11:05.134477+0000","last_clean":"2024-03-13T08:11:05.134477+0000","last_became_active":"2024-03-13T07:34:47.400882+0000","last_became_peered":"2024-03-13T07:34:47.400882+0000","last_unstale":"2024-03-13T08:11:05.134477+0000","last_undegraded":"2024-03-13T08:11:05.134477+0000","last_fullsized":"2024-03-13T08:11:05.134477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:34.136507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:34.136507+0000","last_clean_scrub_stamp":"2024-03-13T07:34:34.136507+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:24:34.761011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":2507,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.190817+0000","last_change":"2024-03-13T07:34:52.336528+0000","last_active":"2024-03-13T08:11:06.190817+0000","last_peered":"2024-03-13T08:11:06.190817+0000","last_clean":"2024-03-13T08:11:06.190817+0000","last_became_active":"2024-03-13T07:34:52.331245+0000","last_became_peered":"2024-03-13T07:34:52.331245+0000","last_unstale":"2024-03-13T08:11:06.190817+0000","last_undegraded":"2024-03-13T08:11:06.190817+0000","last_fullsized":"2024-03-13T08:11:06.190817+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:06:57.704465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":2486,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.244712+0000","last_change":"2024-03-13T07:35:56.278016+0000","last_active":"2024-03-13T08:11:07.244712+0000","last_peered":"2024-03-13T08:11:07.244712+0000","last_clean":"2024-03-13T08:11:07.244712+0000","last_became_active":"2024-03-13T07:35:56.266938+0000","last_became_peered":"2024-03-13T07:35:56.266938+0000","last_unstale":"2024-03-13T08:11:07.244712+0000","last_undegraded":"2024-03-13T08:11:07.244712+0000","last_fullsized":"2024-03-13T08:11:07.244712+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:07:57.842277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2508,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.203456+0000","last_change":"2024-03-13T07:34:52.050005+0000","last_active":"2024-03-13T08:11:06.203456+0000","last_peered":"2024-03-13T08:11:06.203456+0000","last_clean":"2024-03-13T08:11:06.203456+0000","last_became_active":"2024-03-13T07:34:52.021695+0000","last_became_peered":"2024-03-13T07:34:52.021695+0000","last_unstale":"2024-03-13T08:11:06.203456+0000","last_undegraded":"2024-03-13T08:11:06.203456+0000","last_fullsized":"2024-03-13T08:11:06.203456+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:11:09.006656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":2485,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.292538+0000","last_change":"2024-03-13T07:36:13.095056+0000","last_active":"2024-03-13T08:11:07.292538+0000","last_peered":"2024-03-13T08:11:07.292538+0000","last_clean":"2024-03-13T08:11:07.292538+0000","last_became_active":"2024-03-13T07:36:13.088822+0000","last_became_peered":"2024-03-13T07:36:13.088822+0000","last_unstale":"2024-03-13T08:11:07.292538+0000","last_undegraded":"2024-03-13T08:11:07.292538+0000","last_fullsized":"2024-03-13T08:11:07.292538+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T16:26:52.233143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2482,"reported_epoch":1287,"state":"active+clean","last_fresh":"2024-03-13T08:11:04.014501+0000","last_change":"2024-03-13T07:34:52.044522+0000","last_active":"2024-03-13T08:11:04.014501+0000","last_peered":"2024-03-13T08:11:04.014501+0000","last_clean":"2024-03-13T08:11:04.014501+0000","last_became_active":"2024-03-13T07:34:52.014922+0000","last_became_peered":"2024-03-13T07:34:52.014922+0000","last_unstale":"2024-03-13T08:11:04.014501+0000","last_undegraded":"2024-03-13T08:11:04.014501+0000","last_fullsized":"2024-03-13T08:11:04.014501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:59:30.188904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":2490,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.141951+0000","last_change":"2024-03-13T07:36:30.783179+0000","last_active":"2024-03-13T08:11:05.141951+0000","last_peered":"2024-03-13T08:11:05.141951+0000","last_clean":"2024-03-13T08:11:05.141951+0000","last_became_active":"2024-03-13T07:36:30.750203+0000","last_became_peered":"2024-03-13T07:36:30.750203+0000","last_unstale":"2024-03-13T08:11:05.141951+0000","last_undegraded":"2024-03-13T08:11:05.141951+0000","last_fullsized":"2024-03-13T08:11:05.141951+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:16:45.823172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":2507,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.178311+0000","last_change":"2024-03-13T07:34:52.914984+0000","last_active":"2024-03-13T08:11:06.178311+0000","last_peered":"2024-03-13T08:11:06.178311+0000","last_clean":"2024-03-13T08:11:06.178311+0000","last_became_active":"2024-03-13T07:34:52.909827+0000","last_became_peered":"2024-03-13T07:34:52.909827+0000","last_unstale":"2024-03-13T08:11:06.178311+0000","last_undegraded":"2024-03-13T08:11:06.178311+0000","last_fullsized":"2024-03-13T08:11:06.178311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T10:06:44.279005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":2487,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.236325+0000","last_change":"2024-03-13T07:35:55.514723+0000","last_active":"2024-03-13T08:11:07.236325+0000","last_peered":"2024-03-13T08:11:07.236325+0000","last_clean":"2024-03-13T08:11:07.236325+0000","last_became_active":"2024-03-13T07:35:55.507183+0000","last_became_peered":"2024-03-13T07:35:55.507183+0000","last_unstale":"2024-03-13T08:11:07.236325+0000","last_undegraded":"2024-03-13T08:11:07.236325+0000","last_fullsized":"2024-03-13T08:11:07.236325+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T08:32:34.561274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2496,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.278513+0000","last_change":"2024-03-13T07:34:52.930585+0000","last_active":"2024-03-13T08:11:07.278513+0000","last_peered":"2024-03-13T08:11:07.278513+0000","last_clean":"2024-03-13T08:11:07.278513+0000","last_became_active":"2024-03-13T07:34:52.922427+0000","last_became_peered":"2024-03-13T07:34:52.922427+0000","last_unstale":"2024-03-13T08:11:07.278513+0000","last_undegraded":"2024-03-13T08:11:07.278513+0000","last_fullsized":"2024-03-13T08:11:07.278513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T11:07:08.680289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":2482,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.094880+0000","last_change":"2024-03-13T07:36:03.434566+0000","last_active":"2024-03-13T08:11:05.094880+0000","last_peered":"2024-03-13T08:11:05.094880+0000","last_clean":"2024-03-13T08:11:05.094880+0000","last_became_active":"2024-03-13T07:36:03.416513+0000","last_became_peered":"2024-03-13T07:36:03.416513+0000","last_unstale":"2024-03-13T08:11:05.094880+0000","last_undegraded":"2024-03-13T08:11:05.094880+0000","last_fullsized":"2024-03-13T08:11:05.094880+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T09:15:23.095764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2506,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.234296+0000","last_change":"2024-03-13T07:34:52.133418+0000","last_active":"2024-03-13T08:11:06.234296+0000","last_peered":"2024-03-13T08:11:06.234296+0000","last_clean":"2024-03-13T08:11:06.234296+0000","last_became_active":"2024-03-13T07:34:52.128201+0000","last_became_peered":"2024-03-13T07:34:52.128201+0000","last_unstale":"2024-03-13T08:11:06.234296+0000","last_undegraded":"2024-03-13T08:11:06.234296+0000","last_fullsized":"2024-03-13T08:11:06.234296+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T08:41:28.651710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":2482,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.082792+0000","last_change":"2024-03-13T07:35:58.694998+0000","last_active":"2024-03-13T08:11:05.082792+0000","last_peered":"2024-03-13T08:11:05.082792+0000","last_clean":"2024-03-13T08:11:05.082792+0000","last_became_active":"2024-03-13T07:35:58.686006+0000","last_became_peered":"2024-03-13T07:35:58.686006+0000","last_unstale":"2024-03-13T08:11:05.082792+0000","last_undegraded":"2024-03-13T08:11:05.082792+0000","last_fullsized":"2024-03-13T08:11:05.082792+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T11:17:57.352820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2491,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.151354+0000","last_change":"2024-03-13T07:34:52.429709+0000","last_active":"2024-03-13T08:11:05.151354+0000","last_peered":"2024-03-13T08:11:05.151354+0000","last_clean":"2024-03-13T08:11:05.151354+0000","last_became_active":"2024-03-13T07:34:52.404260+0000","last_became_peered":"2024-03-13T07:34:52.404260+0000","last_unstale":"2024-03-13T08:11:05.151354+0000","last_undegraded":"2024-03-13T08:11:05.151354+0000","last_fullsized":"2024-03-13T08:11:05.151354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:01:45.314102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":2496,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.232467+0000","last_change":"2024-03-13T07:36:24.360453+0000","last_active":"2024-03-13T08:11:06.232467+0000","last_peered":"2024-03-13T08:11:06.232467+0000","last_clean":"2024-03-13T08:11:06.232467+0000","last_became_active":"2024-03-13T07:36:24.325717+0000","last_became_peered":"2024-03-13T07:36:24.325717+0000","last_unstale":"2024-03-13T08:11:06.232467+0000","last_undegraded":"2024-03-13T08:11:06.232467+0000","last_fullsized":"2024-03-13T08:11:06.232467+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T09:04:22.666616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":2504,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.262147+0000","last_change":"2024-03-13T07:34:52.389651+0000","last_active":"2024-03-13T08:11:06.262147+0000","last_peered":"2024-03-13T08:11:06.262147+0000","last_clean":"2024-03-13T08:11:06.262147+0000","last_became_active":"2024-03-13T07:34:52.384430+0000","last_became_peered":"2024-03-13T07:34:52.384430+0000","last_unstale":"2024-03-13T08:11:06.262147+0000","last_undegraded":"2024-03-13T08:11:06.262147+0000","last_fullsized":"2024-03-13T08:11:06.262147+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:19:31.011822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":2472,"reported_epoch":1287,"state":"active+clean","last_fresh":"2024-03-13T08:11:04.023402+0000","last_change":"2024-03-13T07:35:57.207526+0000","last_active":"2024-03-13T08:11:04.023402+0000","last_peered":"2024-03-13T08:11:04.023402+0000","last_clean":"2024-03-13T08:11:04.023402+0000","last_became_active":"2024-03-13T07:35:57.201526+0000","last_became_peered":"2024-03-13T07:35:57.201526+0000","last_unstale":"2024-03-13T08:11:04.023402+0000","last_undegraded":"2024-03-13T08:11:04.023402+0000","last_fullsized":"2024-03-13T08:11:04.023402+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T11:45:26.599035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":2487,"reported_epoch":1286,"state":"active+clean","last_fresh":"2024-03-13T08:11:02.498392+0000","last_change":"2024-03-13T07:36:33.047609+0000","last_active":"2024-03-13T08:11:02.498392+0000","last_peered":"2024-03-13T08:11:02.498392+0000","last_clean":"2024-03-13T08:11:02.498392+0000","last_became_active":"2024-03-13T07:36:33.040995+0000","last_became_peered":"2024-03-13T07:36:33.040995+0000","last_unstale":"2024-03-13T08:11:02.498392+0000","last_undegraded":"2024-03-13T08:11:02.498392+0000","last_fullsized":"2024-03-13T08:11:02.498392+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:36:47.927475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":2486,"reported_epoch":1286,"state":"active+clean","last_fresh":"2024-03-13T08:11:02.493814+0000","last_change":"2024-03-13T07:35:59.334344+0000","last_active":"2024-03-13T08:11:02.493814+0000","last_peered":"2024-03-13T08:11:02.493814+0000","last_clean":"2024-03-13T08:11:02.493814+0000","last_became_active":"2024-03-13T07:35:59.327404+0000","last_became_peered":"2024-03-13T07:35:59.327404+0000","last_unstale":"2024-03-13T08:11:02.493814+0000","last_undegraded":"2024-03-13T08:11:02.493814+0000","last_fullsized":"2024-03-13T08:11:02.493814+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T07:51:42.128085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":2485,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.276516+0000","last_change":"2024-03-13T07:35:58.477704+0000","last_active":"2024-03-13T08:11:07.276516+0000","last_peered":"2024-03-13T08:11:07.276516+0000","last_clean":"2024-03-13T08:11:07.276516+0000","last_became_active":"2024-03-13T07:35:58.453584+0000","last_became_peered":"2024-03-13T07:35:58.453584+0000","last_unstale":"2024-03-13T08:11:07.276516+0000","last_undegraded":"2024-03-13T08:11:07.276516+0000","last_fullsized":"2024-03-13T08:11:07.276516+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T11:07:51.798068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":2485,"reported_epoch":1286,"state":"active+clean","last_fresh":"2024-03-13T08:11:02.491482+0000","last_change":"2024-03-13T07:35:58.672119+0000","last_active":"2024-03-13T08:11:02.491482+0000","last_peered":"2024-03-13T08:11:02.491482+0000","last_clean":"2024-03-13T08:11:02.491482+0000","last_became_active":"2024-03-13T07:35:58.650418+0000","last_became_peered":"2024-03-13T07:35:58.650418+0000","last_unstale":"2024-03-13T08:11:02.491482+0000","last_undegraded":"2024-03-13T08:11:02.491482+0000","last_fullsized":"2024-03-13T08:11:02.491482+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:00:23.900254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":2489,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.140968+0000","last_change":"2024-03-13T07:35:58.484442+0000","last_active":"2024-03-13T08:11:05.140968+0000","last_peered":"2024-03-13T08:11:05.140968+0000","last_clean":"2024-03-13T08:11:05.140968+0000","last_became_active":"2024-03-13T07:35:58.196136+0000","last_became_peered":"2024-03-13T07:35:58.196136+0000","last_unstale":"2024-03-13T08:11:05.140968+0000","last_undegraded":"2024-03-13T08:11:05.140968+0000","last_fullsized":"2024-03-13T08:11:05.140968+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T12:03:40.253695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":2482,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.076576+0000","last_change":"2024-03-13T07:36:02.406401+0000","last_active":"2024-03-13T08:11:05.076576+0000","last_peered":"2024-03-13T08:11:05.076576+0000","last_clean":"2024-03-13T08:11:05.076576+0000","last_became_active":"2024-03-13T07:36:01.627175+0000","last_became_peered":"2024-03-13T07:36:01.627175+0000","last_unstale":"2024-03-13T08:11:05.076576+0000","last_undegraded":"2024-03-13T08:11:05.076576+0000","last_fullsized":"2024-03-13T08:11:05.076576+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:33:22.584681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":2489,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.132495+0000","last_change":"2024-03-13T07:35:58.016424+0000","last_active":"2024-03-13T08:11:05.132495+0000","last_peered":"2024-03-13T08:11:05.132495+0000","last_clean":"2024-03-13T08:11:05.132495+0000","last_became_active":"2024-03-13T07:35:57.866652+0000","last_became_peered":"2024-03-13T07:35:57.866652+0000","last_unstale":"2024-03-13T08:11:05.132495+0000","last_undegraded":"2024-03-13T08:11:05.132495+0000","last_fullsized":"2024-03-13T08:11:05.132495+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T07:47:22.030844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":2496,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.174407+0000","last_change":"2024-03-13T07:35:55.780370+0000","last_active":"2024-03-13T08:11:06.174407+0000","last_peered":"2024-03-13T08:11:06.174407+0000","last_clean":"2024-03-13T08:11:06.174407+0000","last_became_active":"2024-03-13T07:35:55.763372+0000","last_became_peered":"2024-03-13T07:35:55.763372+0000","last_unstale":"2024-03-13T08:11:06.174407+0000","last_undegraded":"2024-03-13T08:11:06.174407+0000","last_fullsized":"2024-03-13T08:11:06.174407+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:18:01.324022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":2509,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.120431+0000","last_change":"2024-03-13T07:35:58.780824+0000","last_active":"2024-03-13T08:11:05.120431+0000","last_peered":"2024-03-13T08:11:05.120431+0000","last_clean":"2024-03-13T08:11:05.120431+0000","last_became_active":"2024-03-13T07:35:58.764470+0000","last_became_peered":"2024-03-13T07:35:58.764470+0000","last_unstale":"2024-03-13T08:11:05.120431+0000","last_undegraded":"2024-03-13T08:11:05.120431+0000","last_fullsized":"2024-03-13T08:11:05.120431+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T09:41:57.342177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":2481,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.079458+0000","last_change":"2024-03-13T07:36:03.235996+0000","last_active":"2024-03-13T08:11:05.079458+0000","last_peered":"2024-03-13T08:11:05.079458+0000","last_clean":"2024-03-13T08:11:05.079458+0000","last_became_active":"2024-03-13T07:36:03.227060+0000","last_became_peered":"2024-03-13T07:36:03.227060+0000","last_unstale":"2024-03-13T08:11:05.079458+0000","last_undegraded":"2024-03-13T08:11:05.079458+0000","last_fullsized":"2024-03-13T08:11:05.079458+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T08:43:43.435538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":2486,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.273871+0000","last_change":"2024-03-13T07:35:58.484714+0000","last_active":"2024-03-13T08:11:07.273871+0000","last_peered":"2024-03-13T08:11:07.273871+0000","last_clean":"2024-03-13T08:11:07.273871+0000","last_became_active":"2024-03-13T07:35:58.461765+0000","last_became_peered":"2024-03-13T07:35:58.461765+0000","last_unstale":"2024-03-13T08:11:07.273871+0000","last_undegraded":"2024-03-13T08:11:07.273871+0000","last_fullsized":"2024-03-13T08:11:07.273871+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:31:23.513938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":2495,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.230587+0000","last_change":"2024-03-13T07:36:14.250831+0000","last_active":"2024-03-13T08:11:06.230587+0000","last_peered":"2024-03-13T08:11:06.230587+0000","last_clean":"2024-03-13T08:11:06.230587+0000","last_became_active":"2024-03-13T07:36:14.238576+0000","last_became_peered":"2024-03-13T07:36:14.238576+0000","last_unstale":"2024-03-13T08:11:06.230587+0000","last_undegraded":"2024-03-13T08:11:06.230587+0000","last_fullsized":"2024-03-13T08:11:06.230587+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:40:43.463707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":20,"seq":85899346357,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72052,"kb_used_data":4772,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299788,"statfs":{"total":96636764160,"available":96562982912,"internally_reserved":0,"allocated":4886528,"data_stored":2217715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":20,"seq":85899346358,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72052,"kb_used_data":4772,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299788,"statfs":{"total":96636764160,"available":96562982912,"internally_reserved":0,"allocated":4886528,"data_stored":2217715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":1,"up_from":20,"seq":85899346359,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68260,"kb_used_data":5140,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94303580,"statfs":{"total":96636764160,"available":96566865920,"internally_reserved":0,"allocated":5263360,"data_stored":2600370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[]},{"osd":4,"up_from":18,"seq":77309411771,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72064,"kb_used_data":4776,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299776,"statfs":{"total":96636764160,"available":96562970624,"internally_reserved":0,"allocated":4890624,"data_stored":2217720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719477173,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72056,"kb_used_data":4772,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299784,"statfs":{"total":96636764160,"available":96562978816,"internally_reserved":0,"allocated":4886528,"data_stored":2217715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509877,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72020,"kb_used_data":4736,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299820,"statfs":{"total":96636764160,"available":96563015680,"internally_reserved":0,"allocated":4849664,"data_stored":2194506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834575284,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72016,"kb_used_data":4740,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299824,"statfs":{"total":96636764160,"available":96563019776,"internally_reserved":0,"allocated":4853760,"data_stored":2194511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":68876747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575286,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76628,"kb_used_data":5192,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295212,"statfs":{"total":96636764160,"available":96558297088,"internally_reserved":0,"allocated":5316608,"data_stored":2629850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-13T08:11:09.358 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-13T08:11:09.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.477+0000 7fdc402d0640 10 monclient: get_monmap_and_config 2024-03-13T08:11:09.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.477+0000 7fdc402d0640 10 monclient: build_initial_monmap 2024-03-13T08:11:09.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.477+0000 7fdc402d0640 10 monclient: monmap: 2024-03-13T08:11:09.479 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:09.479 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:09.479 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:09.478707+0000 2024-03-13T08:11:09.479 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:09.478707+0000 2024-03-13T08:11:09.479 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:09.479 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:09.479 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:09.479 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:09.479 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:09.479 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:09.479 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:09.479 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:09.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.478+0000 7fdc402d0640 1 Processor -- start 2024-03-13T08:11:09.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.478+0000 7fdc402d0640 1 -- start start 2024-03-13T08:11:09.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.478+0000 7fdc402d0640 10 monclient: init 2024-03-13T08:11:09.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.479+0000 7fdc402d0640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:09.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.479+0000 7fdc402d0640 10 monclient: _add_conns ranks=[2,4,1,3,0] 2024-03-13T08:11:09.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.479+0000 7fdc402d0640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380b5320 0x7fdc380b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:09.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.479+0000 7fdc402d0640 10 monclient(hunting): picked mon.noname-b con 0x7fdc380b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:09.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.479+0000 7fdc402d0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdc380b1ac0 0x7fdc380b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:09.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.479+0000 7fdc402d0640 10 monclient(hunting): picked mon.noname-c con 0x7fdc380b1ac0 addr v2:172.21.15.195:3300/0 2024-03-13T08:11:09.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.479+0000 7fdc402d0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fdc3813f630 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:09.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.479+0000 7fdc402d0640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc380b1150 con 0x7fdc3813f630 2024-03-13T08:11:09.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.479+0000 7fdc402d0640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:09.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.479+0000 7fdc402d0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc380b5700 con 0x7fdc380b1ac0 2024-03-13T08:11:09.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.479+0000 7fdc402d0640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:09.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.479+0000 7fdc402d0640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380b13c0 con 0x7fdc380b5320 2024-03-13T08:11:09.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.479+0000 7fdc402d0640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:09.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.479+0000 7fdc402d0640 10 monclient(hunting): authenticate will time out at 3496.887542s 2024-03-13T08:11:09.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.479+0000 7fdc3d86b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdc380b1ac0 0x7fdc380b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:09.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.480+0000 7fdc3e06c640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380b5320 0x7fdc380b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:09.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.480+0000 7fdc3d86b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdc380b1ac0 0x7fdc380b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:48894/0 (socket says 172.21.15.132:48894) 2024-03-13T08:11:09.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.480+0000 7fdc3d86b640 1 -- 172.21.15.132:0/853732067 learned_addr learned my addr 172.21.15.132:0/853732067 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:09.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.480+0000 7fdc3d86b640 10 monclient(hunting): get_auth_request con 0x7fdc380b1ac0 auth_method 0 2024-03-13T08:11:09.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.480+0000 7fdc3d86b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:09.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.480+0000 7fdc3d86b640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:09.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.480+0000 7fdc3d86b640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:09.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.480+0000 7fdc3e06c640 10 monclient(hunting): get_auth_request con 0x7fdc380b5320 auth_method 0 2024-03-13T08:11:09.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.480+0000 7fdc3e06c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:09.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.480+0000 7fdc3e06c640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:09.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.480+0000 7fdc3e06c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:09.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.481+0000 7fdc3d86b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:09.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.481+0000 7fdc3d86b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:09.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.481+0000 7fdc3d86b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:09.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.481+0000 7fdc3e06c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:09.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.481+0000 7fdc3e06c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:09.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.481+0000 7fdc3e06c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:09.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.482+0000 7fdc3d86b640 10 monclient(hunting): handle_auth_done global_id 19648 payload 274 2024-03-13T08:11:09.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.482+0000 7fdc3d86b640 1 -- 172.21.15.132:0/853732067 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380b5320 msgr2=0x7fdc380b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:09.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.482+0000 7fdc3d86b640 1 --2- 172.21.15.132:0/853732067 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380b5320 0x7fdc380b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:09.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.482+0000 7fdc3d86b640 1 -- 172.21.15.132:0/853732067 >> v1:172.21.15.195:6789/0 conn(0x7fdc3813f630 legacy=0x7fdc3813fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:09.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.482+0000 7fdc3d86b640 10 monclient: _finish_hunting 0 2024-03-13T08:11:09.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.482+0000 7fdc3d86b640 1 monclient: found mon.noname-c 2024-03-13T08:11:09.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.482+0000 7fdc3d86b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:09.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.482+0000 7fdc3d86b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T08:11:09.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.482+0000 7fdc3d86b640 1 -- 172.21.15.132:0/853732067 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc380b16a0 con 0x7fdc380b1ac0 2024-03-13T08:11:09.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.484+0000 7fdc3d86b640 1 --2- 172.21.15.132:0/853732067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc380b1ac0 0x7fdc380b1f00 secure :-1 s=READY pgs=4723 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009870 tx=0x7fdc2802b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ff2518f3bc86467 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:09.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc3d06a640 1 -- 172.21.15.132:0/853732067 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28038070 con 0x7fdc380b1ac0 2024-03-13T08:11:09.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc3d06a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:09.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc3d06a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T08:11:09.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc3d06a640 10 monclient: dump: 2024-03-13T08:11:09.489 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:09.489 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:09.489 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:09.490 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:09.490 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:09.490 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:09.490 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:09.490 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:09.490 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:09.490 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:09.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc3d06a640 10 monclient: _finish_auth 0 2024-03-13T08:11:09.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc3d06a640 10 monclient: _check_auth_tickets 2024-03-13T08:11:09.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc3d06a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:09.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc3d06a640 1 -- 172.21.15.132:0/853732067 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc28031740 con 0x7fdc380b1ac0 2024-03-13T08:11:09.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc3d06a640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:09.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc3d06a640 1 -- 172.21.15.132:0/853732067 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2802f050 con 0x7fdc380b1ac0 2024-03-13T08:11:09.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc402d0640 5 monclient: authenticate success, global_id 19648 2024-03-13T08:11:09.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc402d0640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:09.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc402d0640 4 set_mon_vals no callback set 2024-03-13T08:11:09.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc3d06a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:09.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc3d06a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:09.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc3d06a640 10 monclient: dump: 2024-03-13T08:11:09.491 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:09.491 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:09.491 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:09.491 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:09.492 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:09.492 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:09.492 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:09.492 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:09.492 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:09.492 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:09.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc402d0640 10 monclient: shutdown 2024-03-13T08:11:09.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc402d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:09.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc402d0640 1 -- 172.21.15.132:0/853732067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc380b1ac0 msgr2=0x7fdc380b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:09.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.489+0000 7fdc402d0640 1 --2- 172.21.15.132:0/853732067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc380b1ac0 0x7fdc380b1f00 secure :-1 s=READY pgs=4723 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009870 tx=0x7fdc2802b7e0 comp rx=0 tx=0).stop 2024-03-13T08:11:09.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.490+0000 7fdc402d0640 1 -- 172.21.15.132:0/853732067 shutdown_connections 2024-03-13T08:11:09.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.490+0000 7fdc402d0640 1 --2- 172.21.15.132:0/853732067 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc380b1ac0 0x7fdc380b1f00 unknown :-1 s=CLOSED pgs=4723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:09.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.490+0000 7fdc402d0640 1 --2- 172.21.15.132:0/853732067 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380b5320 0x7fdc380b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:09.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.490+0000 7fdc402d0640 1 -- 172.21.15.132:0/853732067 >> 172.21.15.132:0/853732067 conn(0x7fdc3806d2b0 msgr2=0x7fdc3806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:09.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.490+0000 7fdc402d0640 1 -- 172.21.15.132:0/853732067 shutdown_connections 2024-03-13T08:11:09.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.490+0000 7fdc402d0640 1 -- 172.21.15.132:0/853732067 wait complete. 2024-03-13T08:11:09.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.490+0000 7fdc402d0640 10 monclient: build_initial_monmap 2024-03-13T08:11:09.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.490+0000 7fdc402d0640 10 monclient: monmap: 2024-03-13T08:11:09.497 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:09.497 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:09.497 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:09.497 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:09.497 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:09.497 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:09.497 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:09.497 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:09.497 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:09.497 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:09.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 1 librados: starting msgr at 2024-03-13T08:11:09.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 1 librados: starting objecter 2024-03-13T08:11:09.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 1 Processor -- start 2024-03-13T08:11:09.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 1 -- start start 2024-03-13T08:11:09.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 1 librados: setting wanted keys 2024-03-13T08:11:09.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 1 librados: calling monclient init 2024-03-13T08:11:09.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 10 monclient: init 2024-03-13T08:11:09.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:09.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 10 monclient: _add_conns ranks=[1,0,2] 2024-03-13T08:11:09.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc380b1ac0 0x7fdc38142e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:09.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 10 monclient(hunting): picked mon.noname-b con 0x7fdc380b1ac0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:09.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc380b5320 0x7fdc3813b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:09.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 10 monclient(hunting): picked mon.noname-a con 0x7fdc380b5320 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:09.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc3813f630 0x7fdc381420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:09.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 10 monclient(hunting): picked mon.noname-c con 0x7fdc3813f630 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:09.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:09.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380b4980 con 0x7fdc380b5320 2024-03-13T08:11:09.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:09.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380b4800 con 0x7fdc380b1ac0 2024-03-13T08:11:09.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:09.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3814cef0 con 0x7fdc3813f630 2024-03-13T08:11:09.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:09.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.491+0000 7fdc402d0640 10 monclient(hunting): authenticate will time out at 3496.899837s 2024-03-13T08:11:09.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.492+0000 7fdc3e06c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc380b1ac0 0x7fdc38142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:09.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.492+0000 7fdc3e86d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc3813f630 0x7fdc381420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:09.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.492+0000 7fdc3e06c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc380b1ac0 0x7fdc38142e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:48908/0 (socket says 172.21.15.132:48908) 2024-03-13T08:11:09.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.492+0000 7fdc3e06c640 1 -- 172.21.15.132:0/3765223443 learned_addr learned my addr 172.21.15.132:0/3765223443 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:09.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.492+0000 7fdc3e06c640 10 monclient(hunting): get_auth_request con 0x7fdc380b1ac0 auth_method 0 2024-03-13T08:11:09.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.492+0000 7fdc3e06c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:09.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.492+0000 7fdc3e06c640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:09.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.492+0000 7fdc3e06c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:09.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.492+0000 7fdc3e86d640 10 monclient(hunting): get_auth_request con 0x7fdc3813f630 auth_method 0 2024-03-13T08:11:09.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.492+0000 7fdc3e86d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:09.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.492+0000 7fdc3e86d640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:09.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.492+0000 7fdc3e86d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:09.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.493+0000 7fdc3e06c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:09.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.493+0000 7fdc3e06c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:09.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.493+0000 7fdc3e06c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:09.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.493+0000 7fdc3e86d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:09.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.493+0000 7fdc3e86d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:09.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.493+0000 7fdc3e86d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:09.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.494+0000 7fdc3d86b640 1 --2- 172.21.15.132:0/3765223443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc380b5320 0x7fdc3813b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:09.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.495+0000 7fdc3e06c640 10 monclient(hunting): handle_auth_done global_id 19651 payload 824 2024-03-13T08:11:09.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.495+0000 7fdc3e06c640 1 -- 172.21.15.132:0/3765223443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc3813f630 msgr2=0x7fdc381420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:09.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.495+0000 7fdc3e06c640 1 --2- 172.21.15.132:0/3765223443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc3813f630 0x7fdc381420d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:09.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.495+0000 7fdc3e06c640 1 -- 172.21.15.132:0/3765223443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc380b5320 msgr2=0x7fdc3813b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:09.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.495+0000 7fdc3e06c640 1 --2- 172.21.15.132:0/3765223443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc380b5320 0x7fdc3813b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:09.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.495+0000 7fdc3e06c640 10 monclient: _finish_hunting 0 2024-03-13T08:11:09.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.495+0000 7fdc3e06c640 1 monclient: found mon.noname-b 2024-03-13T08:11:09.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.495+0000 7fdc3e06c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:09.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.495+0000 7fdc3e06c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T08:11:09.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.495+0000 7fdc3e06c640 1 -- 172.21.15.132:0/3765223443 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc38142960 con 0x7fdc380b1ac0 2024-03-13T08:11:09.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.498+0000 7fdc3e06c640 1 --2- 172.21.15.132:0/3765223443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc380b1ac0 0x7fdc38142e80 secure :-1 s=READY pgs=4724 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400e9b0 tx=0x7fdc3400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:09.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.498+0000 7fdc2effd640 10 client.?.objecter ms_handle_connect 0x7fdc380b1ac0 2024-03-13T08:11:09.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.498+0000 7fdc2effd640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:09.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc2effd640 1 -- 172.21.15.132:0/3765223443 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34017070 con 0x7fdc380b1ac0 2024-03-13T08:11:09.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc2effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:09.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc2effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:09.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc2effd640 10 monclient: dump: 2024-03-13T08:11:09.505 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:09.505 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:09.505 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:09.505 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:09.505 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:09.505 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:09.505 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:09.506 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:09.506 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:09.506 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:09.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc2effd640 10 monclient: _finish_auth 0 2024-03-13T08:11:09.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc2effd640 10 monclient: _check_auth_tickets 2024-03-13T08:11:09.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc2effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:09.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc2effd640 1 -- 172.21.15.132:0/3765223443 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3400c9a0 con 0x7fdc380b1ac0 2024-03-13T08:11:09.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc2effd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:09.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc2effd640 1 -- 172.21.15.132:0/3765223443 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3400cd90 con 0x7fdc380b1ac0 2024-03-13T08:11:09.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc402d0640 5 monclient: authenticate success, global_id 19651 2024-03-13T08:11:09.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc402d0640 10 monclient: _renew_subs 2024-03-13T08:11:09.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc402d0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:09.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc402d0640 1 -- 172.21.15.132:0/3765223443 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc38142610 con 0x7fdc380b1ac0 2024-03-13T08:11:09.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc3c869640 4 set_mon_vals no callback set 2024-03-13T08:11:09.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc2effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:09.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc2effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:09.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc2effd640 10 monclient: dump: 2024-03-13T08:11:09.507 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:09.507 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:09.507 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:09.507 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:09.507 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:09.507 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:09.507 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:09.507 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:09.507 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:09.507 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:09.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc402d0640 10 client.19651.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:09.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc402d0640 10 monclient: _renew_subs 2024-03-13T08:11:09.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc402d0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:09.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc402d0640 1 -- 172.21.15.132:0/3765223443 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc381d7ab0 con 0x7fdc380b1ac0 2024-03-13T08:11:09.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.503+0000 7fdc402d0640 1 librados: init done 2024-03-13T08:11:09.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.504+0000 7fdc402d0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:09.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.504+0000 7fdc402d0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:09.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.504+0000 7fdc402d0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:09.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.504+0000 7fdc402d0640 1 -- 172.21.15.132:0/3765223443 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3804f000 con 0x7fdc380b1ac0 2024-03-13T08:11:09.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.748+0000 7fdc2effd640 1 -- 172.21.15.132:0/3765223443 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fdc3401a070 con 0x7fdc380b1ac0 2024-03-13T08:11:09.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.748+0000 7fdc2effd640 1 --2- 172.21.15.132:0/3765223443 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fdc1c06d560 0x7fdc1c06fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:09.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.749+0000 7fdc3d86b640 1 --2- 172.21.15.132:0/3765223443 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fdc1c06d560 0x7fdc1c06fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:09.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.749+0000 7fdc3d86b640 10 monclient: get_auth_request con 0x7fdc1c06d560 auth_method 0 2024-03-13T08:11:09.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.749+0000 7fdc3d86b640 1 --2- 172.21.15.132:0/3765223443 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fdc1c06d560 0x7fdc1c06fa20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdc28035000 tx=0x7fdc28008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:09.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.749+0000 7fdc2effd640 10 client.19651.objecter ms_handle_connect 0x7fdc1c06d560 2024-03-13T08:11:09.752 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.752+0000 7fdc2effd640 1 -- 172.21.15.132:0/3765223443 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1292..1292 src has 1008..1292) v4 ==== 6305+0+0 (secure 0 0 0) 0x7fdc340577a0 con 0x7fdc380b1ac0 2024-03-13T08:11:09.752 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.752+0000 7fdc2effd640 10 client.19651.objecter ms_dispatch 0x7fdc38000b90 osd_map(1292..1292 src has 1008..1292) v4 2024-03-13T08:11:09.752 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.752+0000 7fdc2effd640 3 client.19651.objecter handle_osd_map got epochs [1292,1292] > 0 2024-03-13T08:11:09.752 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.752+0000 7fdc2effd640 3 client.19651.objecter handle_osd_map decoding full epoch 1292 2024-03-13T08:11:09.752 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.752+0000 7fdc2effd640 20 client.19651.objecter dump_active .. 0 homeless 2024-03-13T08:11:09.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.763+0000 7fdc2effd640 1 -- 172.21.15.132:0/3765223443 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fdc34059700 con 0x7fdc380b1ac0 2024-03-13T08:11:09.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.763+0000 7fdc2effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:09.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.763+0000 7fdc2effd640 10 monclient: _finish_command 1 = system:0 2024-03-13T08:11:09.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.935+0000 7fdc402d0640 1 -- 172.21.15.132:0/3765223443 --> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc381d5520 con 0x7fdc1c06d560 2024-03-13T08:11:09.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.938+0000 7fdc2effd640 1 -- 172.21.15.132:0/3765223443 <== mgr.4104 v2:172.21.15.132:6800/3847550149 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+116241 (secure 0 0 0) 0x7fdc381d5520 con 0x7fdc1c06d560 2024-03-13T08:11:09.939 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T08:11:09.940 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-03-13T08:11:09.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 10 librados: watch_flush enter 2024-03-13T08:11:09.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 10 librados: watch_flush exit 2024-03-13T08:11:09.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 20 client.19651.objecter shutdown clearing up homeless session... 2024-03-13T08:11:09.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 10 client.19651.objecter successfully canceled tick 2024-03-13T08:11:09.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 1 -- 172.21.15.132:0/3765223443 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fdc1c06d560 msgr2=0x7fdc1c06fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:09.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 1 --2- 172.21.15.132:0/3765223443 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fdc1c06d560 0x7fdc1c06fa20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdc28035000 tx=0x7fdc28008030 comp rx=0 tx=0).stop 2024-03-13T08:11:09.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 10 monclient: shutdown 2024-03-13T08:11:09.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:09.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 1 -- 172.21.15.132:0/3765223443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc380b1ac0 msgr2=0x7fdc38142e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:09.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 1 --2- 172.21.15.132:0/3765223443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc380b1ac0 0x7fdc38142e80 secure :-1 s=READY pgs=4724 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400e9b0 tx=0x7fdc3400ee70 comp rx=0 tx=0).stop 2024-03-13T08:11:09.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 1 -- 172.21.15.132:0/3765223443 shutdown_connections 2024-03-13T08:11:09.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 1 --2- 172.21.15.132:0/3765223443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc3813f630 0x7fdc381420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:09.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 1 --2- 172.21.15.132:0/3765223443 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fdc1c06d560 0x7fdc1c06fa20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:09.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 1 --2- 172.21.15.132:0/3765223443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc380b5320 0x7fdc3813b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:09.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 1 --2- 172.21.15.132:0/3765223443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc380b1ac0 0x7fdc38142e80 unknown :-1 s=CLOSED pgs=4724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:09.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 1 -- 172.21.15.132:0/3765223443 >> 172.21.15.132:0/3765223443 conn(0x7fdc3806d2b0 msgr2=0x7fdc381388e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:09.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 1 -- 172.21.15.132:0/3765223443 shutdown_connections 2024-03-13T08:11:09.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 1 -- 172.21.15.132:0/3765223443 wait complete. 2024-03-13T08:11:09.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:09.940+0000 7fdc402d0640 1 librados: shutdown 2024-03-13T08:11:09.952 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":2394,"stamp":"2024-03-13T08:11:09.790587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":91,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":577296,"kb_used_data":39052,"kb_used_omap":12,"kb_used_meta":538099,"kb_avail":754397424,"statfs":{"total":773094113280,"available":772502962176,"internally_reserved":0,"allocated":39989248,"data_stored":18580076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":551013938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":145,"apply_latency_ms":145,"commit_latency_ns":145000000,"apply_latency_ns":145000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.717375"},"pg_stats":[{"pgid":"9.19","version":"0'0","reported_seq":2496,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.195988+0000","last_change":"2024-03-13T07:35:58.099771+0000","last_active":"2024-03-13T08:11:08.195988+0000","last_peered":"2024-03-13T08:11:08.195988+0000","last_clean":"2024-03-13T08:11:08.195988+0000","last_became_active":"2024-03-13T07:35:58.062655+0000","last_became_peered":"2024-03-13T07:35:58.062655+0000","last_unstale":"2024-03-13T08:11:08.195988+0000","last_undegraded":"2024-03-13T08:11:08.195988+0000","last_fullsized":"2024-03-13T08:11:08.195988+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:18:10.709550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":2481,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.069963+0000","last_change":"2024-03-13T07:35:56.417609+0000","last_active":"2024-03-13T08:11:05.069963+0000","last_peered":"2024-03-13T08:11:05.069963+0000","last_clean":"2024-03-13T08:11:05.069963+0000","last_became_active":"2024-03-13T07:35:56.409964+0000","last_became_peered":"2024-03-13T07:35:56.409964+0000","last_unstale":"2024-03-13T08:11:05.069963+0000","last_undegraded":"2024-03-13T08:11:05.069963+0000","last_fullsized":"2024-03-13T08:11:05.069963+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T07:57:49.564938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":2481,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.140120+0000","last_change":"2024-03-13T07:35:59.115829+0000","last_active":"2024-03-13T08:11:05.140120+0000","last_peered":"2024-03-13T08:11:05.140120+0000","last_clean":"2024-03-13T08:11:05.140120+0000","last_became_active":"2024-03-13T07:35:59.109130+0000","last_became_peered":"2024-03-13T07:35:59.109130+0000","last_unstale":"2024-03-13T08:11:05.140120+0000","last_undegraded":"2024-03-13T08:11:05.140120+0000","last_fullsized":"2024-03-13T08:11:05.140120+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:21:21.572511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":2485,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.267971+0000","last_change":"2024-03-13T07:36:26.745752+0000","last_active":"2024-03-13T08:11:07.267971+0000","last_peered":"2024-03-13T08:11:07.267971+0000","last_clean":"2024-03-13T08:11:07.267971+0000","last_became_active":"2024-03-13T07:36:26.737387+0000","last_became_peered":"2024-03-13T07:36:26.737387+0000","last_unstale":"2024-03-13T08:11:07.267971+0000","last_undegraded":"2024-03-13T08:11:07.267971+0000","last_fullsized":"2024-03-13T08:11:07.267971+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:06:10.706337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":2488,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.344331+0000","last_change":"2024-03-13T07:35:56.978987+0000","last_active":"2024-03-13T08:11:07.344331+0000","last_peered":"2024-03-13T08:11:07.344331+0000","last_clean":"2024-03-13T08:11:07.344331+0000","last_became_active":"2024-03-13T07:35:56.973580+0000","last_became_peered":"2024-03-13T07:35:56.973580+0000","last_unstale":"2024-03-13T08:11:07.344331+0000","last_undegraded":"2024-03-13T08:11:07.344331+0000","last_fullsized":"2024-03-13T08:11:07.344331+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:11:56.932620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":2483,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.384809+0000","last_change":"2024-03-13T07:35:56.488106+0000","last_active":"2024-03-13T08:11:08.384809+0000","last_peered":"2024-03-13T08:11:08.384809+0000","last_clean":"2024-03-13T08:11:08.384809+0000","last_became_active":"2024-03-13T07:35:56.482614+0000","last_became_peered":"2024-03-13T07:35:56.482614+0000","last_unstale":"2024-03-13T08:11:08.384809+0000","last_undegraded":"2024-03-13T08:11:08.384809+0000","last_fullsized":"2024-03-13T08:11:08.384809+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:43:18.546715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":2483,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.097675+0000","last_change":"2024-03-13T07:35:56.873267+0000","last_active":"2024-03-13T08:11:05.097675+0000","last_peered":"2024-03-13T08:11:05.097675+0000","last_clean":"2024-03-13T08:11:05.097675+0000","last_became_active":"2024-03-13T07:35:56.864351+0000","last_became_peered":"2024-03-13T07:35:56.864351+0000","last_unstale":"2024-03-13T08:11:05.097675+0000","last_undegraded":"2024-03-13T08:11:05.097675+0000","last_fullsized":"2024-03-13T08:11:05.097675+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T16:48:28.626092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":2483,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.085867+0000","last_change":"2024-03-13T07:35:55.887861+0000","last_active":"2024-03-13T08:11:05.085867+0000","last_peered":"2024-03-13T08:11:05.085867+0000","last_clean":"2024-03-13T08:11:05.085867+0000","last_became_active":"2024-03-13T07:35:55.874083+0000","last_became_peered":"2024-03-13T07:35:55.874083+0000","last_unstale":"2024-03-13T08:11:05.085867+0000","last_undegraded":"2024-03-13T08:11:05.085867+0000","last_fullsized":"2024-03-13T08:11:05.085867+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:55:42.390559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"44'7","reported_seq":2496,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.388540+0000","last_change":"2024-03-13T07:35:58.311435+0000","last_active":"2024-03-13T08:11:08.388540+0000","last_peered":"2024-03-13T08:11:08.388540+0000","last_clean":"2024-03-13T08:11:08.388540+0000","last_became_active":"2024-03-13T07:35:58.303418+0000","last_became_peered":"2024-03-13T07:35:58.303418+0000","last_unstale":"2024-03-13T08:11:08.388540+0000","last_undegraded":"2024-03-13T08:11:08.388540+0000","last_fullsized":"2024-03-13T08:11:08.388540+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:15:47.397228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":2487,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.240610+0000","last_change":"2024-03-13T07:36:00.388143+0000","last_active":"2024-03-13T08:11:07.240610+0000","last_peered":"2024-03-13T08:11:07.240610+0000","last_clean":"2024-03-13T08:11:07.240610+0000","last_became_active":"2024-03-13T07:36:00.379147+0000","last_became_peered":"2024-03-13T07:36:00.379147+0000","last_unstale":"2024-03-13T08:11:07.240610+0000","last_undegraded":"2024-03-13T08:11:07.240610+0000","last_fullsized":"2024-03-13T08:11:07.240610+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:55:32.801526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":2511,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.097762+0000","last_change":"2024-03-13T07:35:55.903931+0000","last_active":"2024-03-13T08:11:05.097762+0000","last_peered":"2024-03-13T08:11:05.097762+0000","last_clean":"2024-03-13T08:11:05.097762+0000","last_became_active":"2024-03-13T07:35:55.898189+0000","last_became_peered":"2024-03-13T07:35:55.898189+0000","last_unstale":"2024-03-13T08:11:05.097762+0000","last_undegraded":"2024-03-13T08:11:05.097762+0000","last_fullsized":"2024-03-13T08:11:05.097762+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T12:46:32.909808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":2498,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.206944+0000","last_change":"2024-03-13T07:35:59.285198+0000","last_active":"2024-03-13T08:11:08.206944+0000","last_peered":"2024-03-13T08:11:08.206944+0000","last_clean":"2024-03-13T08:11:08.206944+0000","last_became_active":"2024-03-13T07:35:59.276875+0000","last_became_peered":"2024-03-13T07:35:59.276875+0000","last_unstale":"2024-03-13T08:11:08.206944+0000","last_undegraded":"2024-03-13T08:11:08.206944+0000","last_fullsized":"2024-03-13T08:11:08.206944+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:08:00.548538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'76","reported_seq":2510,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.134477+0000","last_change":"2024-03-13T07:34:48.043475+0000","last_active":"2024-03-13T08:11:05.134477+0000","last_peered":"2024-03-13T08:11:05.134477+0000","last_clean":"2024-03-13T08:11:05.134477+0000","last_became_active":"2024-03-13T07:34:47.400882+0000","last_became_peered":"2024-03-13T07:34:47.400882+0000","last_unstale":"2024-03-13T08:11:05.134477+0000","last_undegraded":"2024-03-13T08:11:05.134477+0000","last_fullsized":"2024-03-13T08:11:05.134477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:34.136507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:34.136507+0000","last_clean_scrub_stamp":"2024-03-13T07:34:34.136507+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:24:34.761011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":2507,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.190817+0000","last_change":"2024-03-13T07:34:52.336528+0000","last_active":"2024-03-13T08:11:06.190817+0000","last_peered":"2024-03-13T08:11:06.190817+0000","last_clean":"2024-03-13T08:11:06.190817+0000","last_became_active":"2024-03-13T07:34:52.331245+0000","last_became_peered":"2024-03-13T07:34:52.331245+0000","last_unstale":"2024-03-13T08:11:06.190817+0000","last_undegraded":"2024-03-13T08:11:06.190817+0000","last_fullsized":"2024-03-13T08:11:06.190817+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:06:57.704465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":2486,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.244712+0000","last_change":"2024-03-13T07:35:56.278016+0000","last_active":"2024-03-13T08:11:07.244712+0000","last_peered":"2024-03-13T08:11:07.244712+0000","last_clean":"2024-03-13T08:11:07.244712+0000","last_became_active":"2024-03-13T07:35:56.266938+0000","last_became_peered":"2024-03-13T07:35:56.266938+0000","last_unstale":"2024-03-13T08:11:07.244712+0000","last_undegraded":"2024-03-13T08:11:07.244712+0000","last_fullsized":"2024-03-13T08:11:07.244712+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:07:57.842277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2508,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.203456+0000","last_change":"2024-03-13T07:34:52.050005+0000","last_active":"2024-03-13T08:11:06.203456+0000","last_peered":"2024-03-13T08:11:06.203456+0000","last_clean":"2024-03-13T08:11:06.203456+0000","last_became_active":"2024-03-13T07:34:52.021695+0000","last_became_peered":"2024-03-13T07:34:52.021695+0000","last_unstale":"2024-03-13T08:11:06.203456+0000","last_undegraded":"2024-03-13T08:11:06.203456+0000","last_fullsized":"2024-03-13T08:11:06.203456+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:11:09.006656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":2485,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.292538+0000","last_change":"2024-03-13T07:36:13.095056+0000","last_active":"2024-03-13T08:11:07.292538+0000","last_peered":"2024-03-13T08:11:07.292538+0000","last_clean":"2024-03-13T08:11:07.292538+0000","last_became_active":"2024-03-13T07:36:13.088822+0000","last_became_peered":"2024-03-13T07:36:13.088822+0000","last_unstale":"2024-03-13T08:11:07.292538+0000","last_undegraded":"2024-03-13T08:11:07.292538+0000","last_fullsized":"2024-03-13T08:11:07.292538+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T16:26:52.233143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2492,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.390567+0000","last_change":"2024-03-13T07:34:52.044522+0000","last_active":"2024-03-13T08:11:08.390567+0000","last_peered":"2024-03-13T08:11:08.390567+0000","last_clean":"2024-03-13T08:11:08.390567+0000","last_became_active":"2024-03-13T07:34:52.014922+0000","last_became_peered":"2024-03-13T07:34:52.014922+0000","last_unstale":"2024-03-13T08:11:08.390567+0000","last_undegraded":"2024-03-13T08:11:08.390567+0000","last_fullsized":"2024-03-13T08:11:08.390567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:59:30.188904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":2490,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.141951+0000","last_change":"2024-03-13T07:36:30.783179+0000","last_active":"2024-03-13T08:11:05.141951+0000","last_peered":"2024-03-13T08:11:05.141951+0000","last_clean":"2024-03-13T08:11:05.141951+0000","last_became_active":"2024-03-13T07:36:30.750203+0000","last_became_peered":"2024-03-13T07:36:30.750203+0000","last_unstale":"2024-03-13T08:11:05.141951+0000","last_undegraded":"2024-03-13T08:11:05.141951+0000","last_fullsized":"2024-03-13T08:11:05.141951+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:16:45.823172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":2507,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.178311+0000","last_change":"2024-03-13T07:34:52.914984+0000","last_active":"2024-03-13T08:11:06.178311+0000","last_peered":"2024-03-13T08:11:06.178311+0000","last_clean":"2024-03-13T08:11:06.178311+0000","last_became_active":"2024-03-13T07:34:52.909827+0000","last_became_peered":"2024-03-13T07:34:52.909827+0000","last_unstale":"2024-03-13T08:11:06.178311+0000","last_undegraded":"2024-03-13T08:11:06.178311+0000","last_fullsized":"2024-03-13T08:11:06.178311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T10:06:44.279005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":2487,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.236325+0000","last_change":"2024-03-13T07:35:55.514723+0000","last_active":"2024-03-13T08:11:07.236325+0000","last_peered":"2024-03-13T08:11:07.236325+0000","last_clean":"2024-03-13T08:11:07.236325+0000","last_became_active":"2024-03-13T07:35:55.507183+0000","last_became_peered":"2024-03-13T07:35:55.507183+0000","last_unstale":"2024-03-13T08:11:07.236325+0000","last_undegraded":"2024-03-13T08:11:07.236325+0000","last_fullsized":"2024-03-13T08:11:07.236325+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T08:32:34.561274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2496,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.278513+0000","last_change":"2024-03-13T07:34:52.930585+0000","last_active":"2024-03-13T08:11:07.278513+0000","last_peered":"2024-03-13T08:11:07.278513+0000","last_clean":"2024-03-13T08:11:07.278513+0000","last_became_active":"2024-03-13T07:34:52.922427+0000","last_became_peered":"2024-03-13T07:34:52.922427+0000","last_unstale":"2024-03-13T08:11:07.278513+0000","last_undegraded":"2024-03-13T08:11:07.278513+0000","last_fullsized":"2024-03-13T08:11:07.278513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T11:07:08.680289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":2482,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.094880+0000","last_change":"2024-03-13T07:36:03.434566+0000","last_active":"2024-03-13T08:11:05.094880+0000","last_peered":"2024-03-13T08:11:05.094880+0000","last_clean":"2024-03-13T08:11:05.094880+0000","last_became_active":"2024-03-13T07:36:03.416513+0000","last_became_peered":"2024-03-13T07:36:03.416513+0000","last_unstale":"2024-03-13T08:11:05.094880+0000","last_undegraded":"2024-03-13T08:11:05.094880+0000","last_fullsized":"2024-03-13T08:11:05.094880+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T09:15:23.095764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2506,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.234296+0000","last_change":"2024-03-13T07:34:52.133418+0000","last_active":"2024-03-13T08:11:06.234296+0000","last_peered":"2024-03-13T08:11:06.234296+0000","last_clean":"2024-03-13T08:11:06.234296+0000","last_became_active":"2024-03-13T07:34:52.128201+0000","last_became_peered":"2024-03-13T07:34:52.128201+0000","last_unstale":"2024-03-13T08:11:06.234296+0000","last_undegraded":"2024-03-13T08:11:06.234296+0000","last_fullsized":"2024-03-13T08:11:06.234296+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T08:41:28.651710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":2482,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.082792+0000","last_change":"2024-03-13T07:35:58.694998+0000","last_active":"2024-03-13T08:11:05.082792+0000","last_peered":"2024-03-13T08:11:05.082792+0000","last_clean":"2024-03-13T08:11:05.082792+0000","last_became_active":"2024-03-13T07:35:58.686006+0000","last_became_peered":"2024-03-13T07:35:58.686006+0000","last_unstale":"2024-03-13T08:11:05.082792+0000","last_undegraded":"2024-03-13T08:11:05.082792+0000","last_fullsized":"2024-03-13T08:11:05.082792+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T11:17:57.352820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2491,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.151354+0000","last_change":"2024-03-13T07:34:52.429709+0000","last_active":"2024-03-13T08:11:05.151354+0000","last_peered":"2024-03-13T08:11:05.151354+0000","last_clean":"2024-03-13T08:11:05.151354+0000","last_became_active":"2024-03-13T07:34:52.404260+0000","last_became_peered":"2024-03-13T07:34:52.404260+0000","last_unstale":"2024-03-13T08:11:05.151354+0000","last_undegraded":"2024-03-13T08:11:05.151354+0000","last_fullsized":"2024-03-13T08:11:05.151354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:01:45.314102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":2496,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.232467+0000","last_change":"2024-03-13T07:36:24.360453+0000","last_active":"2024-03-13T08:11:06.232467+0000","last_peered":"2024-03-13T08:11:06.232467+0000","last_clean":"2024-03-13T08:11:06.232467+0000","last_became_active":"2024-03-13T07:36:24.325717+0000","last_became_peered":"2024-03-13T07:36:24.325717+0000","last_unstale":"2024-03-13T08:11:06.232467+0000","last_undegraded":"2024-03-13T08:11:06.232467+0000","last_fullsized":"2024-03-13T08:11:06.232467+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T09:04:22.666616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":2504,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.262147+0000","last_change":"2024-03-13T07:34:52.389651+0000","last_active":"2024-03-13T08:11:06.262147+0000","last_peered":"2024-03-13T08:11:06.262147+0000","last_clean":"2024-03-13T08:11:06.262147+0000","last_became_active":"2024-03-13T07:34:52.384430+0000","last_became_peered":"2024-03-13T07:34:52.384430+0000","last_unstale":"2024-03-13T08:11:06.262147+0000","last_undegraded":"2024-03-13T08:11:06.262147+0000","last_fullsized":"2024-03-13T08:11:06.262147+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:19:31.011822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":2482,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.382931+0000","last_change":"2024-03-13T07:35:57.207526+0000","last_active":"2024-03-13T08:11:08.382931+0000","last_peered":"2024-03-13T08:11:08.382931+0000","last_clean":"2024-03-13T08:11:08.382931+0000","last_became_active":"2024-03-13T07:35:57.201526+0000","last_became_peered":"2024-03-13T07:35:57.201526+0000","last_unstale":"2024-03-13T08:11:08.382931+0000","last_undegraded":"2024-03-13T08:11:08.382931+0000","last_fullsized":"2024-03-13T08:11:08.382931+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T11:45:26.599035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":2498,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.217777+0000","last_change":"2024-03-13T07:36:33.047609+0000","last_active":"2024-03-13T08:11:08.217777+0000","last_peered":"2024-03-13T08:11:08.217777+0000","last_clean":"2024-03-13T08:11:08.217777+0000","last_became_active":"2024-03-13T07:36:33.040995+0000","last_became_peered":"2024-03-13T07:36:33.040995+0000","last_unstale":"2024-03-13T08:11:08.217777+0000","last_undegraded":"2024-03-13T08:11:08.217777+0000","last_fullsized":"2024-03-13T08:11:08.217777+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:36:47.927475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":2497,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.215144+0000","last_change":"2024-03-13T07:35:59.334344+0000","last_active":"2024-03-13T08:11:08.215144+0000","last_peered":"2024-03-13T08:11:08.215144+0000","last_clean":"2024-03-13T08:11:08.215144+0000","last_became_active":"2024-03-13T07:35:59.327404+0000","last_became_peered":"2024-03-13T07:35:59.327404+0000","last_unstale":"2024-03-13T08:11:08.215144+0000","last_undegraded":"2024-03-13T08:11:08.215144+0000","last_fullsized":"2024-03-13T08:11:08.215144+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T07:51:42.128085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":2485,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.276516+0000","last_change":"2024-03-13T07:35:58.477704+0000","last_active":"2024-03-13T08:11:07.276516+0000","last_peered":"2024-03-13T08:11:07.276516+0000","last_clean":"2024-03-13T08:11:07.276516+0000","last_became_active":"2024-03-13T07:35:58.453584+0000","last_became_peered":"2024-03-13T07:35:58.453584+0000","last_unstale":"2024-03-13T08:11:07.276516+0000","last_undegraded":"2024-03-13T08:11:07.276516+0000","last_fullsized":"2024-03-13T08:11:07.276516+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T11:07:51.798068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":2496,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.202889+0000","last_change":"2024-03-13T07:35:58.672119+0000","last_active":"2024-03-13T08:11:08.202889+0000","last_peered":"2024-03-13T08:11:08.202889+0000","last_clean":"2024-03-13T08:11:08.202889+0000","last_became_active":"2024-03-13T07:35:58.650418+0000","last_became_peered":"2024-03-13T07:35:58.650418+0000","last_unstale":"2024-03-13T08:11:08.202889+0000","last_undegraded":"2024-03-13T08:11:08.202889+0000","last_fullsized":"2024-03-13T08:11:08.202889+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:00:23.900254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":2489,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.140968+0000","last_change":"2024-03-13T07:35:58.484442+0000","last_active":"2024-03-13T08:11:05.140968+0000","last_peered":"2024-03-13T08:11:05.140968+0000","last_clean":"2024-03-13T08:11:05.140968+0000","last_became_active":"2024-03-13T07:35:58.196136+0000","last_became_peered":"2024-03-13T07:35:58.196136+0000","last_unstale":"2024-03-13T08:11:05.140968+0000","last_undegraded":"2024-03-13T08:11:05.140968+0000","last_fullsized":"2024-03-13T08:11:05.140968+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T12:03:40.253695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":2482,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.076576+0000","last_change":"2024-03-13T07:36:02.406401+0000","last_active":"2024-03-13T08:11:05.076576+0000","last_peered":"2024-03-13T08:11:05.076576+0000","last_clean":"2024-03-13T08:11:05.076576+0000","last_became_active":"2024-03-13T07:36:01.627175+0000","last_became_peered":"2024-03-13T07:36:01.627175+0000","last_unstale":"2024-03-13T08:11:05.076576+0000","last_undegraded":"2024-03-13T08:11:05.076576+0000","last_fullsized":"2024-03-13T08:11:05.076576+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:33:22.584681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":2489,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.132495+0000","last_change":"2024-03-13T07:35:58.016424+0000","last_active":"2024-03-13T08:11:05.132495+0000","last_peered":"2024-03-13T08:11:05.132495+0000","last_clean":"2024-03-13T08:11:05.132495+0000","last_became_active":"2024-03-13T07:35:57.866652+0000","last_became_peered":"2024-03-13T07:35:57.866652+0000","last_unstale":"2024-03-13T08:11:05.132495+0000","last_undegraded":"2024-03-13T08:11:05.132495+0000","last_fullsized":"2024-03-13T08:11:05.132495+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T07:47:22.030844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":2496,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.174407+0000","last_change":"2024-03-13T07:35:55.780370+0000","last_active":"2024-03-13T08:11:06.174407+0000","last_peered":"2024-03-13T08:11:06.174407+0000","last_clean":"2024-03-13T08:11:06.174407+0000","last_became_active":"2024-03-13T07:35:55.763372+0000","last_became_peered":"2024-03-13T07:35:55.763372+0000","last_unstale":"2024-03-13T08:11:06.174407+0000","last_undegraded":"2024-03-13T08:11:06.174407+0000","last_fullsized":"2024-03-13T08:11:06.174407+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:18:01.324022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":2509,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.120431+0000","last_change":"2024-03-13T07:35:58.780824+0000","last_active":"2024-03-13T08:11:05.120431+0000","last_peered":"2024-03-13T08:11:05.120431+0000","last_clean":"2024-03-13T08:11:05.120431+0000","last_became_active":"2024-03-13T07:35:58.764470+0000","last_became_peered":"2024-03-13T07:35:58.764470+0000","last_unstale":"2024-03-13T08:11:05.120431+0000","last_undegraded":"2024-03-13T08:11:05.120431+0000","last_fullsized":"2024-03-13T08:11:05.120431+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T09:41:57.342177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":2481,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.079458+0000","last_change":"2024-03-13T07:36:03.235996+0000","last_active":"2024-03-13T08:11:05.079458+0000","last_peered":"2024-03-13T08:11:05.079458+0000","last_clean":"2024-03-13T08:11:05.079458+0000","last_became_active":"2024-03-13T07:36:03.227060+0000","last_became_peered":"2024-03-13T07:36:03.227060+0000","last_unstale":"2024-03-13T08:11:05.079458+0000","last_undegraded":"2024-03-13T08:11:05.079458+0000","last_fullsized":"2024-03-13T08:11:05.079458+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T08:43:43.435538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":2486,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.273871+0000","last_change":"2024-03-13T07:35:58.484714+0000","last_active":"2024-03-13T08:11:07.273871+0000","last_peered":"2024-03-13T08:11:07.273871+0000","last_clean":"2024-03-13T08:11:07.273871+0000","last_became_active":"2024-03-13T07:35:58.461765+0000","last_became_peered":"2024-03-13T07:35:58.461765+0000","last_unstale":"2024-03-13T08:11:07.273871+0000","last_undegraded":"2024-03-13T08:11:07.273871+0000","last_fullsized":"2024-03-13T08:11:07.273871+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:31:23.513938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":2495,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.230587+0000","last_change":"2024-03-13T07:36:14.250831+0000","last_active":"2024-03-13T08:11:06.230587+0000","last_peered":"2024-03-13T08:11:06.230587+0000","last_clean":"2024-03-13T08:11:06.230587+0000","last_became_active":"2024-03-13T07:36:14.238576+0000","last_became_peered":"2024-03-13T07:36:14.238576+0000","last_unstale":"2024-03-13T08:11:06.230587+0000","last_undegraded":"2024-03-13T08:11:06.230587+0000","last_fullsized":"2024-03-13T08:11:06.230587+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:40:43.463707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":20,"seq":85899346357,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72052,"kb_used_data":4772,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299788,"statfs":{"total":96636764160,"available":96562982912,"internally_reserved":0,"allocated":4886528,"data_stored":2217715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":20,"seq":85899346358,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72052,"kb_used_data":4772,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299788,"statfs":{"total":96636764160,"available":96562982912,"internally_reserved":0,"allocated":4886528,"data_stored":2217715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":1,"up_from":20,"seq":85899346359,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68260,"kb_used_data":5140,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94303580,"statfs":{"total":96636764160,"available":96566865920,"internally_reserved":0,"allocated":5263360,"data_stored":2600370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[]},{"osd":4,"up_from":18,"seq":77309411771,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72064,"kb_used_data":4776,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299776,"statfs":{"total":96636764160,"available":96562970624,"internally_reserved":0,"allocated":4890624,"data_stored":2217720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719477173,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72056,"kb_used_data":4772,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299784,"statfs":{"total":96636764160,"available":96562978816,"internally_reserved":0,"allocated":4886528,"data_stored":2217715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509878,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72092,"kb_used_data":4812,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299748,"statfs":{"total":96636764160,"available":96562941952,"internally_reserved":0,"allocated":4927488,"data_stored":2239493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834575285,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72092,"kb_used_data":4816,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299748,"statfs":{"total":96636764160,"available":96562941952,"internally_reserved":0,"allocated":4931584,"data_stored":2239498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":68876747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575286,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76628,"kb_used_data":5192,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295212,"statfs":{"total":96636764160,"available":96558297088,"internally_reserved":0,"allocated":5316608,"data_stored":2629850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-13T08:11:09.954 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-13T08:11:09.954 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-13T08:11:10.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.062+0000 7f70a414a640 10 monclient: get_monmap_and_config 2024-03-13T08:11:10.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.062+0000 7f70a414a640 10 monclient: build_initial_monmap 2024-03-13T08:11:10.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.062+0000 7f70a414a640 10 monclient: monmap: 2024-03-13T08:11:10.063 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:10.063 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:10.063 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:10.063407+0000 2024-03-13T08:11:10.063 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:10.063407+0000 2024-03-13T08:11:10.063 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:10.063 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:10.063 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:10.063 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:10.064 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:10.064 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:10.064 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:10.064 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:10.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.063+0000 7f70a414a640 1 Processor -- start 2024-03-13T08:11:10.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.063+0000 7f70a414a640 1 -- start start 2024-03-13T08:11:10.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.063+0000 7f70a414a640 10 monclient: init 2024-03-13T08:11:10.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.063+0000 7f70a414a640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:10.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.063+0000 7f70a414a640 10 monclient: _add_conns ranks=[2,1,3,4,0] 2024-03-13T08:11:10.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.064+0000 7f70a414a640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f709c0b1e70 0x7f709c0b2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:10.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.064+0000 7f70a414a640 10 monclient(hunting): picked mon.noname-b con 0x7f709c0b1e70 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:10.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.064+0000 7f70a414a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f709c0b00f0 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:10.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.064+0000 7f70a414a640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f709c142ad0 0x7f709c142f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:10.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.064+0000 7f70a414a640 10 monclient(hunting): picked mon.noname-a con 0x7f709c142ad0 addr v2:172.21.15.132:3300/0 2024-03-13T08:11:10.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.064+0000 7f70a414a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709c0b1390 con 0x7f709c0b00f0 2024-03-13T08:11:10.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.064+0000 7f70a414a640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:10.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.064+0000 7f70a414a640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709c0b1670 con 0x7f709c142ad0 2024-03-13T08:11:10.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.064+0000 7f70a414a640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:10.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.064+0000 7f70a414a640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c0b05a0 con 0x7f709c0b1e70 2024-03-13T08:11:10.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.064+0000 7f70a414a640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:10.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.064+0000 7f70a414a640 10 monclient(hunting): authenticate will time out at 3497.472181s 2024-03-13T08:11:10.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.064+0000 7f70a1ee6640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f709c0b1e70 0x7f709c0b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:10.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.065+0000 7f70a1ee6640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f709c0b1e70 0x7f709c0b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:54650/0 (socket says 172.21.15.132:54650) 2024-03-13T08:11:10.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.065+0000 7f70a1ee6640 1 -- 172.21.15.132:0/644761151 learned_addr learned my addr 172.21.15.132:0/644761151 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:10.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.065+0000 7f70a1ee6640 10 monclient(hunting): get_auth_request con 0x7f709c0b1e70 auth_method 0 2024-03-13T08:11:10.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.065+0000 7f70a1ee6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:10.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.065+0000 7f70a1ee6640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:10.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.065+0000 7f70a1ee6640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:10.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.065+0000 7f70a1ee6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:10.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.065+0000 7f70a1ee6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:10.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.065+0000 7f70a1ee6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:10.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.067+0000 7f70a1ee6640 10 monclient(hunting): handle_auth_done global_id 18950 payload 274 2024-03-13T08:11:10.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.067+0000 7f70a1ee6640 1 -- 172.21.15.132:0/644761151 >> v2:172.21.15.132:3300/0 conn(0x7f709c142ad0 msgr2=0x7f709c142f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:10.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.067+0000 7f70a1ee6640 1 --2- 172.21.15.132:0/644761151 >> v2:172.21.15.132:3300/0 conn(0x7f709c142ad0 0x7f709c142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.067+0000 7f70a1ee6640 1 -- 172.21.15.132:0/644761151 >> v1:172.21.15.195:6789/0 conn(0x7f709c0b00f0 legacy=0x7f709c13fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:10.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.067+0000 7f70a1ee6640 10 monclient: _finish_hunting 0 2024-03-13T08:11:10.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.067+0000 7f70a1ee6640 1 monclient: found mon.noname-b 2024-03-13T08:11:10.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.067+0000 7f70a1ee6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:10.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.067+0000 7f70a1ee6640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T08:11:10.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.067+0000 7f70a1ee6640 1 -- 172.21.15.132:0/644761151 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c0b2790 con 0x7f709c0b1e70 2024-03-13T08:11:10.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.069+0000 7f70a1ee6640 1 --2- 172.21.15.132:0/644761151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f709c0b1e70 0x7f709c0b2250 secure :-1 s=READY pgs=3532 cs=0 l=1 rev1=1 crypto rx=0x7f709c06a210 tx=0x7f709802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8299ea388395673 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:10.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a0ee4640 1 -- 172.21.15.132:0/644761151 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098035020 con 0x7f709c0b1e70 2024-03-13T08:11:10.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a0ee4640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:10.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a0ee4640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:10.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a0ee4640 10 monclient: dump: 2024-03-13T08:11:10.078 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:10.078 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:10.079 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:10.079 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:10.079 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:10.079 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:10.079 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:10.079 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:10.079 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:10.079 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:10.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a0ee4640 10 monclient: _finish_auth 0 2024-03-13T08:11:10.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a0ee4640 10 monclient: _check_auth_tickets 2024-03-13T08:11:10.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a0ee4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:10.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a0ee4640 1 -- 172.21.15.132:0/644761151 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7098031640 con 0x7f709c0b1e70 2024-03-13T08:11:10.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a0ee4640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:10.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a0ee4640 1 -- 172.21.15.132:0/644761151 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709802d040 con 0x7f709c0b1e70 2024-03-13T08:11:10.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a414a640 5 monclient: authenticate success, global_id 18950 2024-03-13T08:11:10.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a414a640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:10.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a414a640 4 set_mon_vals no callback set 2024-03-13T08:11:10.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a0ee4640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:10.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a0ee4640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T08:11:10.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a0ee4640 10 monclient: dump: 2024-03-13T08:11:10.081 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:10.081 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:10.081 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:10.081 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:10.081 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:10.081 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:10.081 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:10.081 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:10.081 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:10.081 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:10.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a414a640 10 monclient: shutdown 2024-03-13T08:11:10.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a414a640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:10.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a414a640 1 -- 172.21.15.132:0/644761151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f709c0b1e70 msgr2=0x7f709c0b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:10.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a414a640 1 --2- 172.21.15.132:0/644761151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f709c0b1e70 0x7f709c0b2250 secure :-1 s=READY pgs=3532 cs=0 l=1 rev1=1 crypto rx=0x7f709c06a210 tx=0x7f709802b8f0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a414a640 1 -- 172.21.15.132:0/644761151 shutdown_connections 2024-03-13T08:11:10.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a414a640 1 --2- 172.21.15.132:0/644761151 >> v2:172.21.15.132:3300/0 conn(0x7f709c142ad0 0x7f709c142f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.073+0000 7f70a414a640 1 --2- 172.21.15.132:0/644761151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f709c0b1e70 0x7f709c0b2250 unknown :-1 s=CLOSED pgs=3532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.074+0000 7f70a414a640 1 -- 172.21.15.132:0/644761151 >> 172.21.15.132:0/644761151 conn(0x7f709c0b80e0 msgr2=0x7f709c0b84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:10.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.074+0000 7f70a414a640 1 -- 172.21.15.132:0/644761151 shutdown_connections 2024-03-13T08:11:10.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.074+0000 7f70a414a640 1 -- 172.21.15.132:0/644761151 wait complete. 2024-03-13T08:11:10.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.074+0000 7f70a414a640 10 monclient: build_initial_monmap 2024-03-13T08:11:10.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.074+0000 7f70a414a640 10 monclient: monmap: 2024-03-13T08:11:10.083 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:10.083 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:10.083 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:10.083 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:10.083 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:10.083 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:10.083 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:10.083 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:10.083 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:10.083 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:10.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 1 librados: starting msgr at 2024-03-13T08:11:10.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 1 librados: starting objecter 2024-03-13T08:11:10.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 1 Processor -- start 2024-03-13T08:11:10.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 1 -- start start 2024-03-13T08:11:10.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 1 librados: setting wanted keys 2024-03-13T08:11:10.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 1 librados: calling monclient init 2024-03-13T08:11:10.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 10 monclient: init 2024-03-13T08:11:10.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:10.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T08:11:10.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f709c0b00f0 0x7f709c13aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:10.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 10 monclient(hunting): picked mon.noname-a con 0x7f709c0b00f0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:10.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f709c0b1e70 0x7f709c13f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:10.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 10 monclient(hunting): picked mon.noname-c con 0x7f709c0b1e70 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:10.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f709c142ad0 0x7f709c13cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:10.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 10 monclient(hunting): picked mon.noname-b con 0x7f709c142ad0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:10.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:10.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c1435d0 con 0x7f709c0b00f0 2024-03-13T08:11:10.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:10.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c143450 con 0x7f709c142ad0 2024-03-13T08:11:10.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:10.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c143750 con 0x7f709c0b1e70 2024-03-13T08:11:10.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:10.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.075+0000 7f70a414a640 10 monclient(hunting): authenticate will time out at 3497.483889s 2024-03-13T08:11:10.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.076+0000 7f70a16e5640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f709c0b1e70 0x7f709c13f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:10.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.076+0000 7f70a16e5640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f709c0b1e70 0x7f709c13f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:54664/0 (socket says 172.21.15.132:54664) 2024-03-13T08:11:10.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.076+0000 7f70a16e5640 1 -- 172.21.15.132:0/2090316569 learned_addr learned my addr 172.21.15.132:0/2090316569 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:10.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.076+0000 7f70a16e5640 10 monclient(hunting): get_auth_request con 0x7f709c0b1e70 auth_method 0 2024-03-13T08:11:10.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.076+0000 7f70a16e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:10.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.076+0000 7f70a16e5640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:10.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.076+0000 7f70a16e5640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:10.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.077+0000 7f70a16e5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:10.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.077+0000 7f70a16e5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:10.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.077+0000 7f70a16e5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:10.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.080+0000 7f70a16e5640 10 monclient(hunting): handle_auth_done global_id 18953 payload 824 2024-03-13T08:11:10.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.080+0000 7f70a16e5640 1 -- 172.21.15.132:0/2090316569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f709c142ad0 msgr2=0x7f709c13cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:10.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.080+0000 7f70a16e5640 1 --2- 172.21.15.132:0/2090316569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f709c142ad0 0x7f709c13cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.080+0000 7f70a16e5640 1 -- 172.21.15.132:0/2090316569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f709c0b00f0 msgr2=0x7f709c13aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:10.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.080+0000 7f70a16e5640 1 --2- 172.21.15.132:0/2090316569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f709c0b00f0 0x7f709c13aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.080+0000 7f70a16e5640 10 monclient: _finish_hunting 0 2024-03-13T08:11:10.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.080+0000 7f70a16e5640 1 monclient: found mon.noname-c 2024-03-13T08:11:10.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.080+0000 7f70a16e5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:10.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.080+0000 7f70a16e5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T08:11:10.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.080+0000 7f70a16e5640 1 -- 172.21.15.132:0/2090316569 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c0a9c50 con 0x7f709c0b1e70 2024-03-13T08:11:10.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.082+0000 7f70a16e5640 1 --2- 172.21.15.132:0/2090316569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f709c0b1e70 0x7f709c13f860 secure :-1 s=READY pgs=3533 cs=0 l=1 rev1=1 crypto rx=0x7f708c00c400 tx=0x7f708c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:10.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.082+0000 7f70927fc640 10 client.?.objecter ms_handle_connect 0x7f709c0b1e70 2024-03-13T08:11:10.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.082+0000 7f70927fc640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:10.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70927fc640 1 -- 172.21.15.132:0/2090316569 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c017070 con 0x7f709c0b1e70 2024-03-13T08:11:10.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:10.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70927fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T08:11:10.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70927fc640 10 monclient: dump: 2024-03-13T08:11:10.090 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:10.091 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:10.091 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:10.092 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:10.093 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:10.093 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:10.093 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:10.093 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:10.093 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:10.093 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:10.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70927fc640 10 monclient: _finish_auth 0 2024-03-13T08:11:10.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70927fc640 10 monclient: _check_auth_tickets 2024-03-13T08:11:10.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:10.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70927fc640 1 -- 172.21.15.132:0/2090316569 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708c0035f0 con 0x7f709c0b1e70 2024-03-13T08:11:10.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70a414a640 5 monclient: authenticate success, global_id 18953 2024-03-13T08:11:10.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70927fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:10.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70927fc640 1 -- 172.21.15.132:0/2090316569 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c003a20 con 0x7f709c0b1e70 2024-03-13T08:11:10.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70a414a640 10 monclient: _renew_subs 2024-03-13T08:11:10.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70a414a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:10.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70a414a640 1 -- 172.21.15.132:0/2090316569 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709c0a9f30 con 0x7f709c0b1e70 2024-03-13T08:11:10.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:10.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70927fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T08:11:10.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70927fc640 10 monclient: dump: 2024-03-13T08:11:10.094 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:10.094 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:10.094 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:10.094 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:10.095 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:10.095 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:10.095 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:10.095 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:10.096 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:10.096 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:10.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f7093fff640 4 set_mon_vals no callback set 2024-03-13T08:11:10.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70a414a640 10 client.18953.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:10.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70a414a640 10 monclient: _renew_subs 2024-03-13T08:11:10.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70a414a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:10.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70a414a640 1 -- 172.21.15.132:0/2090316569 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709c13ed40 con 0x7f709c0b1e70 2024-03-13T08:11:10.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.085+0000 7f70a414a640 1 librados: init done 2024-03-13T08:11:10.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.087+0000 7f70a414a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:10.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.087+0000 7f70a414a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:10.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.087+0000 7f70a414a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:10.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.087+0000 7f70a414a640 1 -- 172.21.15.132:0/2090316569 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709c0aa320 con 0x7f709c0b1e70 2024-03-13T08:11:10.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.108+0000 7f70927fc640 1 -- 172.21.15.132:0/2090316569 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f708c003bc0 con 0x7f709c0b1e70 2024-03-13T08:11:10.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.108+0000 7f70927fc640 1 --2- 172.21.15.132:0/2090316569 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f708006d560 0x7f708006fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:10.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.109+0000 7f70a1ee6640 1 --2- 172.21.15.132:0/2090316569 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f708006d560 0x7f708006fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:10.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.109+0000 7f70a1ee6640 10 monclient: get_auth_request con 0x7f708006d560 auth_method 0 2024-03-13T08:11:10.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.109+0000 7f70927fc640 10 client.18953.objecter ms_handle_connect 0x7f708006d560 2024-03-13T08:11:10.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.109+0000 7f70a1ee6640 1 --2- 172.21.15.132:0/2090316569 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f708006d560 0x7f708006fa20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7098036000 tx=0x7f7098030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:10.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.111+0000 7f70927fc640 1 -- 172.21.15.132:0/2090316569 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(1292..1292 src has 1008..1292) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f708c056800 con 0x7f709c0b1e70 2024-03-13T08:11:10.111 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.111+0000 7f70927fc640 10 client.18953.objecter ms_dispatch 0x7f709c000b90 osd_map(1292..1292 src has 1008..1292) v4 2024-03-13T08:11:10.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.111+0000 7f70927fc640 3 client.18953.objecter handle_osd_map got epochs [1292,1292] > 0 2024-03-13T08:11:10.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.111+0000 7f70927fc640 3 client.18953.objecter handle_osd_map decoding full epoch 1292 2024-03-13T08:11:10.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.111+0000 7f70927fc640 20 client.18953.objecter dump_active .. 0 homeless 2024-03-13T08:11:10.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.360+0000 7f70927fc640 1 -- 172.21.15.132:0/2090316569 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f708c058760 con 0x7f709c0b1e70 2024-03-13T08:11:10.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.360+0000 7f70927fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:10.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.360+0000 7f70927fc640 10 monclient: _finish_command 1 = system:0 2024-03-13T08:11:10.552 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.551+0000 7f70a414a640 1 -- 172.21.15.132:0/2090316569 --> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f709c04f000 con 0x7f708006d560 2024-03-13T08:11:10.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.557+0000 7f70927fc640 1 -- 172.21.15.132:0/2090316569 <== mgr.4104 v2:172.21.15.132:6800/3847550149 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+116241 (secure 0 0 0) 0x7f709c04f000 con 0x7f708006d560 2024-03-13T08:11:10.558 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T08:11:10.559 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-03-13T08:11:10.561 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 10 librados: watch_flush enter 2024-03-13T08:11:10.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 10 librados: watch_flush exit 2024-03-13T08:11:10.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 20 client.18953.objecter shutdown clearing up homeless session... 2024-03-13T08:11:10.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 10 client.18953.objecter successfully canceled tick 2024-03-13T08:11:10.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 1 -- 172.21.15.132:0/2090316569 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f708006d560 msgr2=0x7f708006fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:10.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 1 --2- 172.21.15.132:0/2090316569 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f708006d560 0x7f708006fa20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7098036000 tx=0x7f7098030000 comp rx=0 tx=0).stop 2024-03-13T08:11:10.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 10 monclient: shutdown 2024-03-13T08:11:10.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:10.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 1 -- 172.21.15.132:0/2090316569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f709c0b1e70 msgr2=0x7f709c13f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:10.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 1 --2- 172.21.15.132:0/2090316569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f709c0b1e70 0x7f709c13f860 secure :-1 s=READY pgs=3533 cs=0 l=1 rev1=1 crypto rx=0x7f708c00c400 tx=0x7f708c00c8c0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 1 -- 172.21.15.132:0/2090316569 shutdown_connections 2024-03-13T08:11:10.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 1 --2- 172.21.15.132:0/2090316569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f709c142ad0 0x7f709c13cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 1 --2- 172.21.15.132:0/2090316569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f709c0b1e70 0x7f709c13f860 unknown :-1 s=CLOSED pgs=3533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 1 --2- 172.21.15.132:0/2090316569 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f708006d560 0x7f708006fa20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 1 --2- 172.21.15.132:0/2090316569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f709c0b00f0 0x7f709c13aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 1 -- 172.21.15.132:0/2090316569 >> 172.21.15.132:0/2090316569 conn(0x7f709c0b80e0 msgr2=0x7f709c0b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:10.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 1 -- 172.21.15.132:0/2090316569 shutdown_connections 2024-03-13T08:11:10.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 1 -- 172.21.15.132:0/2090316569 wait complete. 2024-03-13T08:11:10.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.559+0000 7f70a414a640 1 librados: shutdown 2024-03-13T08:11:10.571 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":2395,"stamp":"2024-03-13T08:11:10.049647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":91,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":577296,"kb_used_data":39052,"kb_used_omap":12,"kb_used_meta":538099,"kb_avail":754397424,"statfs":{"total":773094113280,"available":772502962176,"internally_reserved":0,"allocated":39989248,"data_stored":18580076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":551013938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":145,"apply_latency_ms":145,"commit_latency_ns":145000000,"apply_latency_ns":145000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.259872"},"pg_stats":[{"pgid":"9.19","version":"0'0","reported_seq":2496,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.195988+0000","last_change":"2024-03-13T07:35:58.099771+0000","last_active":"2024-03-13T08:11:08.195988+0000","last_peered":"2024-03-13T08:11:08.195988+0000","last_clean":"2024-03-13T08:11:08.195988+0000","last_became_active":"2024-03-13T07:35:58.062655+0000","last_became_peered":"2024-03-13T07:35:58.062655+0000","last_unstale":"2024-03-13T08:11:08.195988+0000","last_undegraded":"2024-03-13T08:11:08.195988+0000","last_fullsized":"2024-03-13T08:11:08.195988+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:18:10.709550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":2481,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.069963+0000","last_change":"2024-03-13T07:35:56.417609+0000","last_active":"2024-03-13T08:11:05.069963+0000","last_peered":"2024-03-13T08:11:05.069963+0000","last_clean":"2024-03-13T08:11:05.069963+0000","last_became_active":"2024-03-13T07:35:56.409964+0000","last_became_peered":"2024-03-13T07:35:56.409964+0000","last_unstale":"2024-03-13T08:11:05.069963+0000","last_undegraded":"2024-03-13T08:11:05.069963+0000","last_fullsized":"2024-03-13T08:11:05.069963+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T07:57:49.564938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":2481,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.140120+0000","last_change":"2024-03-13T07:35:59.115829+0000","last_active":"2024-03-13T08:11:05.140120+0000","last_peered":"2024-03-13T08:11:05.140120+0000","last_clean":"2024-03-13T08:11:05.140120+0000","last_became_active":"2024-03-13T07:35:59.109130+0000","last_became_peered":"2024-03-13T07:35:59.109130+0000","last_unstale":"2024-03-13T08:11:05.140120+0000","last_undegraded":"2024-03-13T08:11:05.140120+0000","last_fullsized":"2024-03-13T08:11:05.140120+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:21:21.572511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":2485,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.267971+0000","last_change":"2024-03-13T07:36:26.745752+0000","last_active":"2024-03-13T08:11:07.267971+0000","last_peered":"2024-03-13T08:11:07.267971+0000","last_clean":"2024-03-13T08:11:07.267971+0000","last_became_active":"2024-03-13T07:36:26.737387+0000","last_became_peered":"2024-03-13T07:36:26.737387+0000","last_unstale":"2024-03-13T08:11:07.267971+0000","last_undegraded":"2024-03-13T08:11:07.267971+0000","last_fullsized":"2024-03-13T08:11:07.267971+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:06:10.706337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":2488,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.344331+0000","last_change":"2024-03-13T07:35:56.978987+0000","last_active":"2024-03-13T08:11:07.344331+0000","last_peered":"2024-03-13T08:11:07.344331+0000","last_clean":"2024-03-13T08:11:07.344331+0000","last_became_active":"2024-03-13T07:35:56.973580+0000","last_became_peered":"2024-03-13T07:35:56.973580+0000","last_unstale":"2024-03-13T08:11:07.344331+0000","last_undegraded":"2024-03-13T08:11:07.344331+0000","last_fullsized":"2024-03-13T08:11:07.344331+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:11:56.932620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":2483,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.384809+0000","last_change":"2024-03-13T07:35:56.488106+0000","last_active":"2024-03-13T08:11:08.384809+0000","last_peered":"2024-03-13T08:11:08.384809+0000","last_clean":"2024-03-13T08:11:08.384809+0000","last_became_active":"2024-03-13T07:35:56.482614+0000","last_became_peered":"2024-03-13T07:35:56.482614+0000","last_unstale":"2024-03-13T08:11:08.384809+0000","last_undegraded":"2024-03-13T08:11:08.384809+0000","last_fullsized":"2024-03-13T08:11:08.384809+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:43:18.546715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":2483,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.097675+0000","last_change":"2024-03-13T07:35:56.873267+0000","last_active":"2024-03-13T08:11:05.097675+0000","last_peered":"2024-03-13T08:11:05.097675+0000","last_clean":"2024-03-13T08:11:05.097675+0000","last_became_active":"2024-03-13T07:35:56.864351+0000","last_became_peered":"2024-03-13T07:35:56.864351+0000","last_unstale":"2024-03-13T08:11:05.097675+0000","last_undegraded":"2024-03-13T08:11:05.097675+0000","last_fullsized":"2024-03-13T08:11:05.097675+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T16:48:28.626092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":2483,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.085867+0000","last_change":"2024-03-13T07:35:55.887861+0000","last_active":"2024-03-13T08:11:05.085867+0000","last_peered":"2024-03-13T08:11:05.085867+0000","last_clean":"2024-03-13T08:11:05.085867+0000","last_became_active":"2024-03-13T07:35:55.874083+0000","last_became_peered":"2024-03-13T07:35:55.874083+0000","last_unstale":"2024-03-13T08:11:05.085867+0000","last_undegraded":"2024-03-13T08:11:05.085867+0000","last_fullsized":"2024-03-13T08:11:05.085867+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:55:42.390559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"44'7","reported_seq":2496,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.388540+0000","last_change":"2024-03-13T07:35:58.311435+0000","last_active":"2024-03-13T08:11:08.388540+0000","last_peered":"2024-03-13T08:11:08.388540+0000","last_clean":"2024-03-13T08:11:08.388540+0000","last_became_active":"2024-03-13T07:35:58.303418+0000","last_became_peered":"2024-03-13T07:35:58.303418+0000","last_unstale":"2024-03-13T08:11:08.388540+0000","last_undegraded":"2024-03-13T08:11:08.388540+0000","last_fullsized":"2024-03-13T08:11:08.388540+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:15:47.397228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":2487,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.240610+0000","last_change":"2024-03-13T07:36:00.388143+0000","last_active":"2024-03-13T08:11:07.240610+0000","last_peered":"2024-03-13T08:11:07.240610+0000","last_clean":"2024-03-13T08:11:07.240610+0000","last_became_active":"2024-03-13T07:36:00.379147+0000","last_became_peered":"2024-03-13T07:36:00.379147+0000","last_unstale":"2024-03-13T08:11:07.240610+0000","last_undegraded":"2024-03-13T08:11:07.240610+0000","last_fullsized":"2024-03-13T08:11:07.240610+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:55:32.801526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":2511,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.097762+0000","last_change":"2024-03-13T07:35:55.903931+0000","last_active":"2024-03-13T08:11:05.097762+0000","last_peered":"2024-03-13T08:11:05.097762+0000","last_clean":"2024-03-13T08:11:05.097762+0000","last_became_active":"2024-03-13T07:35:55.898189+0000","last_became_peered":"2024-03-13T07:35:55.898189+0000","last_unstale":"2024-03-13T08:11:05.097762+0000","last_undegraded":"2024-03-13T08:11:05.097762+0000","last_fullsized":"2024-03-13T08:11:05.097762+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T12:46:32.909808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":2498,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.206944+0000","last_change":"2024-03-13T07:35:59.285198+0000","last_active":"2024-03-13T08:11:08.206944+0000","last_peered":"2024-03-13T08:11:08.206944+0000","last_clean":"2024-03-13T08:11:08.206944+0000","last_became_active":"2024-03-13T07:35:59.276875+0000","last_became_peered":"2024-03-13T07:35:59.276875+0000","last_unstale":"2024-03-13T08:11:08.206944+0000","last_undegraded":"2024-03-13T08:11:08.206944+0000","last_fullsized":"2024-03-13T08:11:08.206944+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:08:00.548538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'76","reported_seq":2510,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.134477+0000","last_change":"2024-03-13T07:34:48.043475+0000","last_active":"2024-03-13T08:11:05.134477+0000","last_peered":"2024-03-13T08:11:05.134477+0000","last_clean":"2024-03-13T08:11:05.134477+0000","last_became_active":"2024-03-13T07:34:47.400882+0000","last_became_peered":"2024-03-13T07:34:47.400882+0000","last_unstale":"2024-03-13T08:11:05.134477+0000","last_undegraded":"2024-03-13T08:11:05.134477+0000","last_fullsized":"2024-03-13T08:11:05.134477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:34.136507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:34.136507+0000","last_clean_scrub_stamp":"2024-03-13T07:34:34.136507+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:24:34.761011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":2507,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.190817+0000","last_change":"2024-03-13T07:34:52.336528+0000","last_active":"2024-03-13T08:11:06.190817+0000","last_peered":"2024-03-13T08:11:06.190817+0000","last_clean":"2024-03-13T08:11:06.190817+0000","last_became_active":"2024-03-13T07:34:52.331245+0000","last_became_peered":"2024-03-13T07:34:52.331245+0000","last_unstale":"2024-03-13T08:11:06.190817+0000","last_undegraded":"2024-03-13T08:11:06.190817+0000","last_fullsized":"2024-03-13T08:11:06.190817+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:06:57.704465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":2486,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.244712+0000","last_change":"2024-03-13T07:35:56.278016+0000","last_active":"2024-03-13T08:11:07.244712+0000","last_peered":"2024-03-13T08:11:07.244712+0000","last_clean":"2024-03-13T08:11:07.244712+0000","last_became_active":"2024-03-13T07:35:56.266938+0000","last_became_peered":"2024-03-13T07:35:56.266938+0000","last_unstale":"2024-03-13T08:11:07.244712+0000","last_undegraded":"2024-03-13T08:11:07.244712+0000","last_fullsized":"2024-03-13T08:11:07.244712+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:07:57.842277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2508,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.203456+0000","last_change":"2024-03-13T07:34:52.050005+0000","last_active":"2024-03-13T08:11:06.203456+0000","last_peered":"2024-03-13T08:11:06.203456+0000","last_clean":"2024-03-13T08:11:06.203456+0000","last_became_active":"2024-03-13T07:34:52.021695+0000","last_became_peered":"2024-03-13T07:34:52.021695+0000","last_unstale":"2024-03-13T08:11:06.203456+0000","last_undegraded":"2024-03-13T08:11:06.203456+0000","last_fullsized":"2024-03-13T08:11:06.203456+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:11:09.006656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":2485,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.292538+0000","last_change":"2024-03-13T07:36:13.095056+0000","last_active":"2024-03-13T08:11:07.292538+0000","last_peered":"2024-03-13T08:11:07.292538+0000","last_clean":"2024-03-13T08:11:07.292538+0000","last_became_active":"2024-03-13T07:36:13.088822+0000","last_became_peered":"2024-03-13T07:36:13.088822+0000","last_unstale":"2024-03-13T08:11:07.292538+0000","last_undegraded":"2024-03-13T08:11:07.292538+0000","last_fullsized":"2024-03-13T08:11:07.292538+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T16:26:52.233143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2492,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.390567+0000","last_change":"2024-03-13T07:34:52.044522+0000","last_active":"2024-03-13T08:11:08.390567+0000","last_peered":"2024-03-13T08:11:08.390567+0000","last_clean":"2024-03-13T08:11:08.390567+0000","last_became_active":"2024-03-13T07:34:52.014922+0000","last_became_peered":"2024-03-13T07:34:52.014922+0000","last_unstale":"2024-03-13T08:11:08.390567+0000","last_undegraded":"2024-03-13T08:11:08.390567+0000","last_fullsized":"2024-03-13T08:11:08.390567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:59:30.188904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":2490,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.141951+0000","last_change":"2024-03-13T07:36:30.783179+0000","last_active":"2024-03-13T08:11:05.141951+0000","last_peered":"2024-03-13T08:11:05.141951+0000","last_clean":"2024-03-13T08:11:05.141951+0000","last_became_active":"2024-03-13T07:36:30.750203+0000","last_became_peered":"2024-03-13T07:36:30.750203+0000","last_unstale":"2024-03-13T08:11:05.141951+0000","last_undegraded":"2024-03-13T08:11:05.141951+0000","last_fullsized":"2024-03-13T08:11:05.141951+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:16:45.823172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":2507,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.178311+0000","last_change":"2024-03-13T07:34:52.914984+0000","last_active":"2024-03-13T08:11:06.178311+0000","last_peered":"2024-03-13T08:11:06.178311+0000","last_clean":"2024-03-13T08:11:06.178311+0000","last_became_active":"2024-03-13T07:34:52.909827+0000","last_became_peered":"2024-03-13T07:34:52.909827+0000","last_unstale":"2024-03-13T08:11:06.178311+0000","last_undegraded":"2024-03-13T08:11:06.178311+0000","last_fullsized":"2024-03-13T08:11:06.178311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T10:06:44.279005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":2487,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.236325+0000","last_change":"2024-03-13T07:35:55.514723+0000","last_active":"2024-03-13T08:11:07.236325+0000","last_peered":"2024-03-13T08:11:07.236325+0000","last_clean":"2024-03-13T08:11:07.236325+0000","last_became_active":"2024-03-13T07:35:55.507183+0000","last_became_peered":"2024-03-13T07:35:55.507183+0000","last_unstale":"2024-03-13T08:11:07.236325+0000","last_undegraded":"2024-03-13T08:11:07.236325+0000","last_fullsized":"2024-03-13T08:11:07.236325+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T08:32:34.561274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2496,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.278513+0000","last_change":"2024-03-13T07:34:52.930585+0000","last_active":"2024-03-13T08:11:07.278513+0000","last_peered":"2024-03-13T08:11:07.278513+0000","last_clean":"2024-03-13T08:11:07.278513+0000","last_became_active":"2024-03-13T07:34:52.922427+0000","last_became_peered":"2024-03-13T07:34:52.922427+0000","last_unstale":"2024-03-13T08:11:07.278513+0000","last_undegraded":"2024-03-13T08:11:07.278513+0000","last_fullsized":"2024-03-13T08:11:07.278513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T11:07:08.680289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":2482,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.094880+0000","last_change":"2024-03-13T07:36:03.434566+0000","last_active":"2024-03-13T08:11:05.094880+0000","last_peered":"2024-03-13T08:11:05.094880+0000","last_clean":"2024-03-13T08:11:05.094880+0000","last_became_active":"2024-03-13T07:36:03.416513+0000","last_became_peered":"2024-03-13T07:36:03.416513+0000","last_unstale":"2024-03-13T08:11:05.094880+0000","last_undegraded":"2024-03-13T08:11:05.094880+0000","last_fullsized":"2024-03-13T08:11:05.094880+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T09:15:23.095764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2506,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.234296+0000","last_change":"2024-03-13T07:34:52.133418+0000","last_active":"2024-03-13T08:11:06.234296+0000","last_peered":"2024-03-13T08:11:06.234296+0000","last_clean":"2024-03-13T08:11:06.234296+0000","last_became_active":"2024-03-13T07:34:52.128201+0000","last_became_peered":"2024-03-13T07:34:52.128201+0000","last_unstale":"2024-03-13T08:11:06.234296+0000","last_undegraded":"2024-03-13T08:11:06.234296+0000","last_fullsized":"2024-03-13T08:11:06.234296+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T08:41:28.651710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":2482,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.082792+0000","last_change":"2024-03-13T07:35:58.694998+0000","last_active":"2024-03-13T08:11:05.082792+0000","last_peered":"2024-03-13T08:11:05.082792+0000","last_clean":"2024-03-13T08:11:05.082792+0000","last_became_active":"2024-03-13T07:35:58.686006+0000","last_became_peered":"2024-03-13T07:35:58.686006+0000","last_unstale":"2024-03-13T08:11:05.082792+0000","last_undegraded":"2024-03-13T08:11:05.082792+0000","last_fullsized":"2024-03-13T08:11:05.082792+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T11:17:57.352820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2491,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.151354+0000","last_change":"2024-03-13T07:34:52.429709+0000","last_active":"2024-03-13T08:11:05.151354+0000","last_peered":"2024-03-13T08:11:05.151354+0000","last_clean":"2024-03-13T08:11:05.151354+0000","last_became_active":"2024-03-13T07:34:52.404260+0000","last_became_peered":"2024-03-13T07:34:52.404260+0000","last_unstale":"2024-03-13T08:11:05.151354+0000","last_undegraded":"2024-03-13T08:11:05.151354+0000","last_fullsized":"2024-03-13T08:11:05.151354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:01:45.314102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":2496,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.232467+0000","last_change":"2024-03-13T07:36:24.360453+0000","last_active":"2024-03-13T08:11:06.232467+0000","last_peered":"2024-03-13T08:11:06.232467+0000","last_clean":"2024-03-13T08:11:06.232467+0000","last_became_active":"2024-03-13T07:36:24.325717+0000","last_became_peered":"2024-03-13T07:36:24.325717+0000","last_unstale":"2024-03-13T08:11:06.232467+0000","last_undegraded":"2024-03-13T08:11:06.232467+0000","last_fullsized":"2024-03-13T08:11:06.232467+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T09:04:22.666616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":2504,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.262147+0000","last_change":"2024-03-13T07:34:52.389651+0000","last_active":"2024-03-13T08:11:06.262147+0000","last_peered":"2024-03-13T08:11:06.262147+0000","last_clean":"2024-03-13T08:11:06.262147+0000","last_became_active":"2024-03-13T07:34:52.384430+0000","last_became_peered":"2024-03-13T07:34:52.384430+0000","last_unstale":"2024-03-13T08:11:06.262147+0000","last_undegraded":"2024-03-13T08:11:06.262147+0000","last_fullsized":"2024-03-13T08:11:06.262147+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:19:31.011822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":2482,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.382931+0000","last_change":"2024-03-13T07:35:57.207526+0000","last_active":"2024-03-13T08:11:08.382931+0000","last_peered":"2024-03-13T08:11:08.382931+0000","last_clean":"2024-03-13T08:11:08.382931+0000","last_became_active":"2024-03-13T07:35:57.201526+0000","last_became_peered":"2024-03-13T07:35:57.201526+0000","last_unstale":"2024-03-13T08:11:08.382931+0000","last_undegraded":"2024-03-13T08:11:08.382931+0000","last_fullsized":"2024-03-13T08:11:08.382931+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T11:45:26.599035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":2498,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.217777+0000","last_change":"2024-03-13T07:36:33.047609+0000","last_active":"2024-03-13T08:11:08.217777+0000","last_peered":"2024-03-13T08:11:08.217777+0000","last_clean":"2024-03-13T08:11:08.217777+0000","last_became_active":"2024-03-13T07:36:33.040995+0000","last_became_peered":"2024-03-13T07:36:33.040995+0000","last_unstale":"2024-03-13T08:11:08.217777+0000","last_undegraded":"2024-03-13T08:11:08.217777+0000","last_fullsized":"2024-03-13T08:11:08.217777+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:36:47.927475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":2497,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.215144+0000","last_change":"2024-03-13T07:35:59.334344+0000","last_active":"2024-03-13T08:11:08.215144+0000","last_peered":"2024-03-13T08:11:08.215144+0000","last_clean":"2024-03-13T08:11:08.215144+0000","last_became_active":"2024-03-13T07:35:59.327404+0000","last_became_peered":"2024-03-13T07:35:59.327404+0000","last_unstale":"2024-03-13T08:11:08.215144+0000","last_undegraded":"2024-03-13T08:11:08.215144+0000","last_fullsized":"2024-03-13T08:11:08.215144+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T07:51:42.128085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":2485,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.276516+0000","last_change":"2024-03-13T07:35:58.477704+0000","last_active":"2024-03-13T08:11:07.276516+0000","last_peered":"2024-03-13T08:11:07.276516+0000","last_clean":"2024-03-13T08:11:07.276516+0000","last_became_active":"2024-03-13T07:35:58.453584+0000","last_became_peered":"2024-03-13T07:35:58.453584+0000","last_unstale":"2024-03-13T08:11:07.276516+0000","last_undegraded":"2024-03-13T08:11:07.276516+0000","last_fullsized":"2024-03-13T08:11:07.276516+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T11:07:51.798068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":2496,"reported_epoch":1292,"state":"active+clean","last_fresh":"2024-03-13T08:11:08.202889+0000","last_change":"2024-03-13T07:35:58.672119+0000","last_active":"2024-03-13T08:11:08.202889+0000","last_peered":"2024-03-13T08:11:08.202889+0000","last_clean":"2024-03-13T08:11:08.202889+0000","last_became_active":"2024-03-13T07:35:58.650418+0000","last_became_peered":"2024-03-13T07:35:58.650418+0000","last_unstale":"2024-03-13T08:11:08.202889+0000","last_undegraded":"2024-03-13T08:11:08.202889+0000","last_fullsized":"2024-03-13T08:11:08.202889+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:00:23.900254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":2489,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.140968+0000","last_change":"2024-03-13T07:35:58.484442+0000","last_active":"2024-03-13T08:11:05.140968+0000","last_peered":"2024-03-13T08:11:05.140968+0000","last_clean":"2024-03-13T08:11:05.140968+0000","last_became_active":"2024-03-13T07:35:58.196136+0000","last_became_peered":"2024-03-13T07:35:58.196136+0000","last_unstale":"2024-03-13T08:11:05.140968+0000","last_undegraded":"2024-03-13T08:11:05.140968+0000","last_fullsized":"2024-03-13T08:11:05.140968+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T12:03:40.253695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":2482,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.076576+0000","last_change":"2024-03-13T07:36:02.406401+0000","last_active":"2024-03-13T08:11:05.076576+0000","last_peered":"2024-03-13T08:11:05.076576+0000","last_clean":"2024-03-13T08:11:05.076576+0000","last_became_active":"2024-03-13T07:36:01.627175+0000","last_became_peered":"2024-03-13T07:36:01.627175+0000","last_unstale":"2024-03-13T08:11:05.076576+0000","last_undegraded":"2024-03-13T08:11:05.076576+0000","last_fullsized":"2024-03-13T08:11:05.076576+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:33:22.584681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":2489,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.132495+0000","last_change":"2024-03-13T07:35:58.016424+0000","last_active":"2024-03-13T08:11:05.132495+0000","last_peered":"2024-03-13T08:11:05.132495+0000","last_clean":"2024-03-13T08:11:05.132495+0000","last_became_active":"2024-03-13T07:35:57.866652+0000","last_became_peered":"2024-03-13T07:35:57.866652+0000","last_unstale":"2024-03-13T08:11:05.132495+0000","last_undegraded":"2024-03-13T08:11:05.132495+0000","last_fullsized":"2024-03-13T08:11:05.132495+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T07:47:22.030844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":2496,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.174407+0000","last_change":"2024-03-13T07:35:55.780370+0000","last_active":"2024-03-13T08:11:06.174407+0000","last_peered":"2024-03-13T08:11:06.174407+0000","last_clean":"2024-03-13T08:11:06.174407+0000","last_became_active":"2024-03-13T07:35:55.763372+0000","last_became_peered":"2024-03-13T07:35:55.763372+0000","last_unstale":"2024-03-13T08:11:06.174407+0000","last_undegraded":"2024-03-13T08:11:06.174407+0000","last_fullsized":"2024-03-13T08:11:06.174407+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:18:01.324022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":2509,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.120431+0000","last_change":"2024-03-13T07:35:58.780824+0000","last_active":"2024-03-13T08:11:05.120431+0000","last_peered":"2024-03-13T08:11:05.120431+0000","last_clean":"2024-03-13T08:11:05.120431+0000","last_became_active":"2024-03-13T07:35:58.764470+0000","last_became_peered":"2024-03-13T07:35:58.764470+0000","last_unstale":"2024-03-13T08:11:05.120431+0000","last_undegraded":"2024-03-13T08:11:05.120431+0000","last_fullsized":"2024-03-13T08:11:05.120431+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T09:41:57.342177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":2481,"reported_epoch":1289,"state":"active+clean","last_fresh":"2024-03-13T08:11:05.079458+0000","last_change":"2024-03-13T07:36:03.235996+0000","last_active":"2024-03-13T08:11:05.079458+0000","last_peered":"2024-03-13T08:11:05.079458+0000","last_clean":"2024-03-13T08:11:05.079458+0000","last_became_active":"2024-03-13T07:36:03.227060+0000","last_became_peered":"2024-03-13T07:36:03.227060+0000","last_unstale":"2024-03-13T08:11:05.079458+0000","last_undegraded":"2024-03-13T08:11:05.079458+0000","last_fullsized":"2024-03-13T08:11:05.079458+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T08:43:43.435538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":2486,"reported_epoch":1291,"state":"active+clean","last_fresh":"2024-03-13T08:11:07.273871+0000","last_change":"2024-03-13T07:35:58.484714+0000","last_active":"2024-03-13T08:11:07.273871+0000","last_peered":"2024-03-13T08:11:07.273871+0000","last_clean":"2024-03-13T08:11:07.273871+0000","last_became_active":"2024-03-13T07:35:58.461765+0000","last_became_peered":"2024-03-13T07:35:58.461765+0000","last_unstale":"2024-03-13T08:11:07.273871+0000","last_undegraded":"2024-03-13T08:11:07.273871+0000","last_fullsized":"2024-03-13T08:11:07.273871+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:31:23.513938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":2495,"reported_epoch":1290,"state":"active+clean","last_fresh":"2024-03-13T08:11:06.230587+0000","last_change":"2024-03-13T07:36:14.250831+0000","last_active":"2024-03-13T08:11:06.230587+0000","last_peered":"2024-03-13T08:11:06.230587+0000","last_clean":"2024-03-13T08:11:06.230587+0000","last_became_active":"2024-03-13T07:36:14.238576+0000","last_became_peered":"2024-03-13T07:36:14.238576+0000","last_unstale":"2024-03-13T08:11:06.230587+0000","last_undegraded":"2024-03-13T08:11:06.230587+0000","last_fullsized":"2024-03-13T08:11:06.230587+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:40:43.463707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":20,"seq":85899346357,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72052,"kb_used_data":4772,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299788,"statfs":{"total":96636764160,"available":96562982912,"internally_reserved":0,"allocated":4886528,"data_stored":2217715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":20,"seq":85899346358,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72052,"kb_used_data":4772,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299788,"statfs":{"total":96636764160,"available":96562982912,"internally_reserved":0,"allocated":4886528,"data_stored":2217715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":1,"up_from":20,"seq":85899346359,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68260,"kb_used_data":5140,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94303580,"statfs":{"total":96636764160,"available":96566865920,"internally_reserved":0,"allocated":5263360,"data_stored":2600370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[]},{"osd":4,"up_from":18,"seq":77309411771,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72064,"kb_used_data":4776,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299776,"statfs":{"total":96636764160,"available":96562970624,"internally_reserved":0,"allocated":4890624,"data_stored":2217720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719477173,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72056,"kb_used_data":4772,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299784,"statfs":{"total":96636764160,"available":96562978816,"internally_reserved":0,"allocated":4886528,"data_stored":2217715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509878,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72092,"kb_used_data":4812,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299748,"statfs":{"total":96636764160,"available":96562941952,"internally_reserved":0,"allocated":4927488,"data_stored":2239493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834575285,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72092,"kb_used_data":4816,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299748,"statfs":{"total":96636764160,"available":96562941952,"internally_reserved":0,"allocated":4931584,"data_stored":2239498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":68876747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575286,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76628,"kb_used_data":5192,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295212,"statfs":{"total":96636764160,"available":96558297088,"internally_reserved":0,"allocated":5316608,"data_stored":2629850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-13T08:11:10.573 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-13T08:11:10.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.676+0000 7f8bfe31c640 10 monclient: get_monmap_and_config 2024-03-13T08:11:10.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.676+0000 7f8bfe31c640 10 monclient: build_initial_monmap 2024-03-13T08:11:10.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.676+0000 7f8bfe31c640 10 monclient: monmap: 2024-03-13T08:11:10.679 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:10.680 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:10.680 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:10.677288+0000 2024-03-13T08:11:10.680 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:10.677288+0000 2024-03-13T08:11:10.680 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:10.680 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:10.680 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:10.680 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:10.680 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:10.680 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:10.680 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:10.680 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:10.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 1 Processor -- start 2024-03-13T08:11:10.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 1 -- start start 2024-03-13T08:11:10.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 10 monclient: init 2024-03-13T08:11:10.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:10.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 10 monclient: _add_conns ranks=[3,2,0,1,4] 2024-03-13T08:11:10.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8bf80ad8e0 0x7f8bf8145af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:10.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 10 monclient(hunting): picked mon.noname-a con 0x7f8bf80ad8e0 addr v2:172.21.15.132:3300/0 2024-03-13T08:11:10.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bf80adcc0 0x7f8bf8146630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:10.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 10 monclient(hunting): picked mon.noname-b con 0x7f8bf80adcc0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:10.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8bf8072a20 addr v1:172.21.15.132:6789/0 2024-03-13T08:11:10.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf8146110 con 0x7f8bf8072a20 2024-03-13T08:11:10.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:10.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf806e630 con 0x7f8bf80ad8e0 2024-03-13T08:11:10.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:10.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80b0350 con 0x7f8bf80adcc0 2024-03-13T08:11:10.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:10.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.677+0000 7f8bfe31c640 10 monclient(hunting): authenticate will time out at 3498.085965s 2024-03-13T08:11:10.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.678+0000 7f8bf77fe640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bf80adcc0 0x7f8bf8146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:10.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.678+0000 7f8bfc8b9640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f8bf8072a20 0x7f8bf806df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:60792/0 (socket says 172.21.15.132:60792) 2024-03-13T08:11:10.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.678+0000 7f8bfc8b9640 1 -- 172.21.15.132:0/4270673699 learned_addr learned my addr 172.21.15.132:0/4270673699 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T08:11:10.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.678+0000 7f8bf7fff640 1 --2- 172.21.15.132:0/4270673699 >> v2:172.21.15.132:3300/0 conn(0x7f8bf80ad8e0 0x7f8bf8145af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:10.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.678+0000 7f8bf77fe640 10 monclient(hunting): get_auth_request con 0x7f8bf80adcc0 auth_method 0 2024-03-13T08:11:10.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.678+0000 7f8bf77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:10.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.678+0000 7f8bf77fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:10.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.678+0000 7f8bf77fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:10.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.679+0000 7f8bf7fff640 10 monclient(hunting): get_auth_request con 0x7f8bf80ad8e0 auth_method 0 2024-03-13T08:11:10.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.679+0000 7f8bf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:10.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.679+0000 7f8bf7fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:10.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.679+0000 7f8bf7fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:10.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.679+0000 7f8bf77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:10.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.679+0000 7f8bf77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:10.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.679+0000 7f8bf77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:10.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.680+0000 7f8bf77fe640 10 monclient(hunting): handle_auth_done global_id 18956 payload 274 2024-03-13T08:11:10.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.681+0000 7f8bf77fe640 1 -- 172.21.15.132:0/4270673699 >> v2:172.21.15.132:3300/0 conn(0x7f8bf80ad8e0 msgr2=0x7f8bf8145af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:10.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.681+0000 7f8bf77fe640 1 --2- 172.21.15.132:0/4270673699 >> v2:172.21.15.132:3300/0 conn(0x7f8bf80ad8e0 0x7f8bf8145af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.681+0000 7f8bf77fe640 1 -- 172.21.15.132:0/4270673699 >> v1:172.21.15.132:6789/0 conn(0x7f8bf8072a20 legacy=0x7f8bf806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:10.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.681+0000 7f8bf77fe640 10 monclient: _finish_hunting 0 2024-03-13T08:11:10.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.681+0000 7f8bf77fe640 1 monclient: found mon.noname-b 2024-03-13T08:11:10.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.681+0000 7f8bf77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:10.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.681+0000 7f8bf77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T08:11:10.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.681+0000 7f8bf77fe640 1 -- 172.21.15.132:0/4270673699 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf813fa60 con 0x7f8bf80adcc0 2024-03-13T08:11:10.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.682+0000 7f8bf77fe640 1 --2- 172.21.15.132:0/4270673699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bf80adcc0 0x7f8bf8146630 secure :-1 s=READY pgs=3534 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0097a0 tx=0x7f8bec02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a8fa1290cfeea86 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:10.695 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bf6ffd640 1 -- 172.21.15.132:0/4270673699 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec035020 con 0x7f8bf80adcc0 2024-03-13T08:11:10.696 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bf6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:10.696 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bf6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:10.696 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bf6ffd640 10 monclient: dump: 2024-03-13T08:11:10.696 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:10.696 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:10.696 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:10.696 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:10.696 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:10.696 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:10.696 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:10.696 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:10.696 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:10.696 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:10.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bf6ffd640 10 monclient: _finish_auth 0 2024-03-13T08:11:10.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bf6ffd640 10 monclient: _check_auth_tickets 2024-03-13T08:11:10.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bf6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:10.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bf6ffd640 1 -- 172.21.15.132:0/4270673699 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec031830 con 0x7f8bf80adcc0 2024-03-13T08:11:10.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bf6ffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:10.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bf6ffd640 1 -- 172.21.15.132:0/4270673699 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec031c00 con 0x7f8bf80adcc0 2024-03-13T08:11:10.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bfe31c640 5 monclient: authenticate success, global_id 18956 2024-03-13T08:11:10.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bfe31c640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:10.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bfe31c640 4 set_mon_vals no callback set 2024-03-13T08:11:10.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bf6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:10.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bf6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T08:11:10.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bf6ffd640 10 monclient: dump: 2024-03-13T08:11:10.697 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:10.697 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:10.698 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:10.698 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:10.698 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:10.698 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:10.698 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:10.698 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:10.698 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:10.698 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:10.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bfe31c640 10 monclient: shutdown 2024-03-13T08:11:10.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bfe31c640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:10.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bfe31c640 1 -- 172.21.15.132:0/4270673699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bf80adcc0 msgr2=0x7f8bf8146630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:10.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bfe31c640 1 --2- 172.21.15.132:0/4270673699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bf80adcc0 0x7f8bf8146630 secure :-1 s=READY pgs=3534 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0097a0 tx=0x7f8bec02b8d0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bfe31c640 1 -- 172.21.15.132:0/4270673699 shutdown_connections 2024-03-13T08:11:10.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bfe31c640 1 --2- 172.21.15.132:0/4270673699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bf80adcc0 0x7f8bf8146630 unknown :-1 s=CLOSED pgs=3534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bfe31c640 1 --2- 172.21.15.132:0/4270673699 >> v2:172.21.15.132:3300/0 conn(0x7f8bf80ad8e0 0x7f8bf8145af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.686+0000 7f8bfe31c640 1 -- 172.21.15.132:0/4270673699 >> 172.21.15.132:0/4270673699 conn(0x7f8bf80b3e20 msgr2=0x7f8bf80b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:10.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.687+0000 7f8bfe31c640 1 -- 172.21.15.132:0/4270673699 shutdown_connections 2024-03-13T08:11:10.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.687+0000 7f8bfe31c640 1 -- 172.21.15.132:0/4270673699 wait complete. 2024-03-13T08:11:10.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.687+0000 7f8bfe31c640 10 monclient: build_initial_monmap 2024-03-13T08:11:10.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.687+0000 7f8bfe31c640 10 monclient: monmap: 2024-03-13T08:11:10.699 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:10.699 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:10.699 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:10.699 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:10.699 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:10.699 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:10.699 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:10.699 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:10.700 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:10.700 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:10.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.687+0000 7f8bfe31c640 1 librados: starting msgr at 2024-03-13T08:11:10.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.687+0000 7f8bfe31c640 1 librados: starting objecter 2024-03-13T08:11:10.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 1 Processor -- start 2024-03-13T08:11:10.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 1 -- start start 2024-03-13T08:11:10.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 1 librados: setting wanted keys 2024-03-13T08:11:10.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 1 librados: calling monclient init 2024-03-13T08:11:10.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 10 monclient: init 2024-03-13T08:11:10.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:10.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 10 monclient: _add_conns ranks=[2,1,0] 2024-03-13T08:11:10.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bf80ad8e0 0x7f8bf813ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:10.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 10 monclient(hunting): picked mon.noname-c con 0x7f8bf80ad8e0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:10.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bf813d670 0x7f8bf813dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:10.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 10 monclient(hunting): picked mon.noname-b con 0x7f8bf813d670 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:10.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bf81dafa0 0x7f8bf81db420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:10.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 10 monclient(hunting): picked mon.noname-a con 0x7f8bf81dafa0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:10.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:10.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf8140210 con 0x7f8bf81dafa0 2024-03-13T08:11:10.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:10.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf8140090 con 0x7f8bf813d670 2024-03-13T08:11:10.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:10.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf814ce80 con 0x7f8bf80ad8e0 2024-03-13T08:11:10.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:10.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.688+0000 7f8bfe31c640 10 monclient(hunting): authenticate will time out at 3498.096934s 2024-03-13T08:11:10.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.689+0000 7f8bfc8b9640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bf81dafa0 0x7f8bf81db420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:10.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.689+0000 7f8bf7fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bf80ad8e0 0x7f8bf813ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:10.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.689+0000 7f8bfc8b9640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bf81dafa0 0x7f8bf81db420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51092/0 (socket says 172.21.15.132:51092) 2024-03-13T08:11:10.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.689+0000 7f8bfc8b9640 1 -- 172.21.15.132:0/2647985201 learned_addr learned my addr 172.21.15.132:0/2647985201 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:10.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.689+0000 7f8bfc8b9640 10 monclient(hunting): get_auth_request con 0x7f8bf81dafa0 auth_method 0 2024-03-13T08:11:10.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.689+0000 7f8bfc8b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:10.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.689+0000 7f8bfc8b9640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:10.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.689+0000 7f8bfc8b9640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:10.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.689+0000 7f8bf7fff640 10 monclient(hunting): get_auth_request con 0x7f8bf80ad8e0 auth_method 0 2024-03-13T08:11:10.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.689+0000 7f8bf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:10.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.689+0000 7f8bf7fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:10.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.689+0000 7f8bf7fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:10.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.689+0000 7f8bf77fe640 1 --2- 172.21.15.132:0/2647985201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bf813d670 0x7f8bf813dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:10.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.690+0000 7f8bfc8b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:10.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.690+0000 7f8bfc8b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:10.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.690+0000 7f8bfc8b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:10.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.690+0000 7f8bf77fe640 10 monclient(hunting): get_auth_request con 0x7f8bf813d670 auth_method 0 2024-03-13T08:11:10.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.690+0000 7f8bf77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:10.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.690+0000 7f8bf77fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:10.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.690+0000 7f8bf77fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:10.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.690+0000 7f8bf7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:10.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.690+0000 7f8bf7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:10.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.690+0000 7f8bf7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:10.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.690+0000 7f8bf77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:10.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.690+0000 7f8bf77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:10.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.690+0000 7f8bf77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:10.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.693+0000 7f8bf77fe640 10 monclient(hunting): handle_auth_done global_id 19654 payload 824 2024-03-13T08:11:10.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.693+0000 7f8bf77fe640 1 -- 172.21.15.132:0/2647985201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bf80ad8e0 msgr2=0x7f8bf813ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:10.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.693+0000 7f8bf77fe640 1 --2- 172.21.15.132:0/2647985201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bf80ad8e0 0x7f8bf813ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.693+0000 7f8bf77fe640 1 -- 172.21.15.132:0/2647985201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bf81dafa0 msgr2=0x7f8bf81db420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:10.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.693+0000 7f8bf77fe640 1 --2- 172.21.15.132:0/2647985201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bf81dafa0 0x7f8bf81db420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:10.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.693+0000 7f8bf77fe640 10 monclient: _finish_hunting 0 2024-03-13T08:11:10.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.693+0000 7f8bf77fe640 1 monclient: found mon.noname-b 2024-03-13T08:11:10.704 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.693+0000 7f8bf77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:10.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.693+0000 7f8bf77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T08:11:10.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.693+0000 7f8bf77fe640 1 -- 172.21.15.132:0/2647985201 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf814c890 con 0x7f8bf813d670 2024-03-13T08:11:10.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.695+0000 7f8bf77fe640 1 --2- 172.21.15.132:0/2647985201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bf813d670 0x7f8bf813dad0 secure :-1 s=READY pgs=4725 cs=0 l=1 rev1=1 crypto rx=0x7f8bec033000 tx=0x7f8bec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:10.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.695+0000 7f8bf4ff9640 10 client.?.objecter ms_handle_connect 0x7f8bf813d670 2024-03-13T08:11:10.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.695+0000 7f8bf4ff9640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:10.706 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bf4ff9640 1 -- 172.21.15.132:0/2647985201 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec040070 con 0x7f8bf813d670 2024-03-13T08:11:10.706 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bf4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:10.706 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bf4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:10.706 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bf4ff9640 10 monclient: dump: 2024-03-13T08:11:10.707 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:10.707 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:10.707 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:10.707 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:10.707 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:10.707 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:10.707 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:10.707 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:10.707 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:10.707 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:10.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bf4ff9640 10 monclient: _finish_auth 0 2024-03-13T08:11:10.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bf4ff9640 10 monclient: _check_auth_tickets 2024-03-13T08:11:10.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bf4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:10.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bf4ff9640 1 -- 172.21.15.132:0/2647985201 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec03e070 con 0x7f8bf813d670 2024-03-13T08:11:10.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bf4ff9640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:10.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bf4ff9640 1 -- 172.21.15.132:0/2647985201 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec035040 con 0x7f8bf813d670 2024-03-13T08:11:10.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bf4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:10.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bf4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:10.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bf4ff9640 10 monclient: dump: 2024-03-13T08:11:10.708 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:10.708 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:10.708 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:10.708 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:10.708 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:10.708 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:10.708 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:10.708 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:10.708 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:10.708 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:10.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bfe31c640 5 monclient: authenticate success, global_id 19654 2024-03-13T08:11:10.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bfe31c640 10 monclient: _renew_subs 2024-03-13T08:11:10.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bfe31c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:10.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bfe31c640 1 -- 172.21.15.132:0/2647985201 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf813e8b0 con 0x7f8bf813d670 2024-03-13T08:11:10.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bf67fc640 4 set_mon_vals no callback set 2024-03-13T08:11:10.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bfe31c640 10 client.19654.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:10.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bfe31c640 10 monclient: _renew_subs 2024-03-13T08:11:10.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bfe31c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:10.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bfe31c640 1 -- 172.21.15.132:0/2647985201 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf813ee70 con 0x7f8bf813d670 2024-03-13T08:11:10.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.699+0000 7f8bfe31c640 1 librados: init done 2024-03-13T08:11:10.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.700+0000 7f8bfe31c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:10.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.700+0000 7f8bfe31c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:10.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.700+0000 7f8bfe31c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:10.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.700+0000 7f8bfe31c640 1 -- 172.21.15.132:0/2647985201 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf804f000 con 0x7f8bf813d670 2024-03-13T08:11:10.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.718+0000 7f8bf4ff9640 1 -- 172.21.15.132:0/2647985201 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f8bec038020 con 0x7f8bf813d670 2024-03-13T08:11:10.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.718+0000 7f8bf4ff9640 1 --2- 172.21.15.132:0/2647985201 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f8be406d560 0x7f8be406fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:10.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.718+0000 7f8bf7fff640 1 --2- 172.21.15.132:0/2647985201 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f8be406d560 0x7f8be406fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:10.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.719+0000 7f8bf7fff640 10 monclient: get_auth_request con 0x7f8be406d560 auth_method 0 2024-03-13T08:11:10.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.719+0000 7f8bf4ff9640 10 client.19654.objecter ms_handle_connect 0x7f8be406d560 2024-03-13T08:11:10.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.719+0000 7f8bf7fff640 1 --2- 172.21.15.132:0/2647985201 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f8be406d560 0x7f8be406fa20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8be8006fd0 tx=0x7f8be8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:10.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.721+0000 7f8bf4ff9640 1 -- 172.21.15.132:0/2647985201 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f8bec0317c0 con 0x7f8bf813d670 2024-03-13T08:11:10.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.721+0000 7f8bf4ff9640 10 client.19654.objecter ms_dispatch 0x7f8bf8000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:10.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.721+0000 7f8bf4ff9640 3 client.19654.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:10.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.721+0000 7f8bf4ff9640 3 client.19654.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:10.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.721+0000 7f8bf4ff9640 20 client.19654.objecter dump_active .. 0 homeless 2024-03-13T08:11:10.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.961+0000 7f8bf4ff9640 1 -- 172.21.15.132:0/2647985201 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f8bec07a520 con 0x7f8bf813d670 2024-03-13T08:11:10.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.961+0000 7f8bf4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:10.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:10.961+0000 7f8bf4ff9640 10 monclient: _finish_command 1 = system:0 2024-03-13T08:11:11.144 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.144+0000 7f8bfe31c640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-03-13T08:11:11.144 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.144+0000 7f8bfe31c640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T08:11:11.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.144+0000 7f8bfe31c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:11.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.144+0000 7f8bfe31c640 1 -- 172.21.15.132:0/2647985201 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bf813d390 con 0x7f8bf813d670 2024-03-13T08:11:11.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.186+0000 7f8bf4ff9640 1 -- 172.21.15.132:0/2647985201 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v1293) v1 ==== 74+0+17699 (secure 0 0 0) 0x7f8bec07e5c0 con 0x7f8bf813d670 2024-03-13T08:11:11.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.186+0000 7f8bf4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-03-13T08:11:11.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.186+0000 7f8bf4ff9640 10 monclient: _finish_command 2 = system:0 2024-03-13T08:11:11.187 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T08:11:11.187 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1293,"fsid":"74e4bc2f-6a5e-4423-a8c8-807970a2f96c","created":"2024-03-13T07:33:47.494293+0000","modified":"2024-03-13T08:11:10.011046+0000","last_up_change":"2024-03-13T07:34:44.296009+0000","last_in_change":"2024-03-13T07:33:53.171597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":119,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":341,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-13T07:34:33.504752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-13T07:34:49.707222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"ceph_test_rados_api_asio","create_time":"2024-03-13T07:35:52.093212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0edb216c-bc1e-4865-aafe-218611f3eb09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":1288,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":590522438},{"type":"v1","addr":"172.21.15.132:6819","nonce":590522438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":590522438},{"type":"v1","addr":"172.21.15.132:6821","nonce":590522438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":590522438},{"type":"v1","addr":"172.21.15.132:6825","nonce":590522438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":590522438},{"type":"v1","addr":"172.21.15.132:6823","nonce":590522438}]},"public_addr":"172.21.15.132:6819/590522438","cluster_addr":"172.21.15.132:6821/590522438","heartbeat_back_addr":"172.21.15.132:6825/590522438","heartbeat_front_addr":"172.21.15.132:6823/590522438","state":["exists","up"]},{"osd":1,"uuid":"6be0ae6b-951e-4279-ba0a-9b05b6e6e85e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":1273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1071068206},{"type":"v1","addr":"172.21.15.132:6803","nonce":1071068206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1071068206},{"type":"v1","addr":"172.21.15.132:6805","nonce":1071068206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":1071068206},{"type":"v1","addr":"172.21.15.132:6809","nonce":1071068206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1071068206},{"type":"v1","addr":"172.21.15.132:6807","nonce":1071068206}]},"public_addr":"172.21.15.132:6803/1071068206","cluster_addr":"172.21.15.132:6805/1071068206","heartbeat_back_addr":"172.21.15.132:6809/1071068206","heartbeat_front_addr":"172.21.15.132:6807/1071068206","state":["exists","up"]},{"osd":2,"uuid":"18008b7f-cfe7-4450-a3cf-aab63ebbe99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":1281,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":958254560},{"type":"v1","addr":"172.21.15.132:6827","nonce":958254560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":958254560},{"type":"v1","addr":"172.21.15.132:6829","nonce":958254560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":958254560},{"type":"v1","addr":"172.21.15.132:6833","nonce":958254560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":958254560},{"type":"v1","addr":"172.21.15.132:6831","nonce":958254560}]},"public_addr":"172.21.15.132:6827/958254560","cluster_addr":"172.21.15.132:6829/958254560","heartbeat_back_addr":"172.21.15.132:6833/958254560","heartbeat_front_addr":"172.21.15.132:6831/958254560","state":["exists","up"]},{"osd":3,"uuid":"411b6066-4b9f-40e7-b92d-4ba914eab63e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":1288,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6811","nonce":1770057764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6813","nonce":1770057764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6817","nonce":1770057764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":1770057764},{"type":"v1","addr":"172.21.15.132:6815","nonce":1770057764}]},"public_addr":"172.21.15.132:6811/1770057764","cluster_addr":"172.21.15.132:6813/1770057764","heartbeat_back_addr":"172.21.15.132:6817/1770057764","heartbeat_front_addr":"172.21.15.132:6815/1770057764","state":["exists","up"]},{"osd":4,"uuid":"1a719246-ae04-42b0-bd9c-fc318dc57ecb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":1288,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":699638467},{"type":"v1","addr":"172.21.15.195:6803","nonce":699638467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":699638467},{"type":"v1","addr":"172.21.15.195:6809","nonce":699638467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":699638467},{"type":"v1","addr":"172.21.15.195:6815","nonce":699638467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6811","nonce":699638467},{"type":"v1","addr":"172.21.15.195:6813","nonce":699638467}]},"public_addr":"172.21.15.195:6803/699638467","cluster_addr":"172.21.15.195:6809/699638467","heartbeat_back_addr":"172.21.15.195:6815/699638467","heartbeat_front_addr":"172.21.15.195:6813/699638467","state":["exists","up"]},{"osd":5,"uuid":"9d9ee991-241b-4507-ba2f-e3b5e2844d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":1288,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6817","nonce":1953907706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6819","nonce":1953907706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6825","nonce":1953907706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6821","nonce":1953907706},{"type":"v1","addr":"172.21.15.195:6822","nonce":1953907706}]},"public_addr":"172.21.15.195:6817/1953907706","cluster_addr":"172.21.15.195:6819/1953907706","heartbeat_back_addr":"172.21.15.195:6825/1953907706","heartbeat_front_addr":"172.21.15.195:6822/1953907706","state":["exists","up"]},{"osd":6,"uuid":"af8b34fd-927b-4860-971b-9fddf699a0dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":1288,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6801","nonce":4106887608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6805","nonce":4106887608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6812","nonce":4106887608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4106887608},{"type":"v1","addr":"172.21.15.195:6808","nonce":4106887608}]},"public_addr":"172.21.15.195:6801/4106887608","cluster_addr":"172.21.15.195:6805/4106887608","heartbeat_back_addr":"172.21.15.195:6812/4106887608","heartbeat_front_addr":"172.21.15.195:6808/4106887608","state":["exists","up"]},{"osd":7,"uuid":"19a39645-8664-463b-bae3-d7aa1026e5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":1288,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6823","nonce":3378300314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6827","nonce":3378300314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6831","nonce":3378300314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3378300314},{"type":"v1","addr":"172.21.15.195:6829","nonce":3378300314}]},"public_addr":"172.21.15.195:6823/3378300314","cluster_addr":"172.21.15.195:6827/3378300314","heartbeat_back_addr":"172.21.15.195:6831/3378300314","heartbeat_front_addr":"172.21.15.195:6829/3378300314","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:32.324299+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:31.786840+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:32.474925+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:31.877950+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:30.195463+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:30.243386+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:30.409185+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T07:34:29.949919+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":3,"length":2}]},{"pool":16,"snaps":[{"begin":5,"length":2}]},{"pool":29,"snaps":[{"begin":1,"length":1}]},{"pool":65,"snaps":[{"begin":1,"length":1}]},{"pool":86,"snaps":[{"begin":7,"length":2}]},{"pool":100,"snaps":[{"begin":1,"length":1}]},{"pool":113,"snaps":[{"begin":1,"length":2}]},{"pool":117,"snaps":[{"begin":3,"length":1}]},{"pool":125,"snaps":[{"begin":2,"length":2}]},{"pool":127,"snaps":[{"begin":3,"length":2}]},{"pool":132,"snaps":[{"begin":2,"length":3}]},{"pool":136,"snaps":[{"begin":2,"length":1}]},{"pool":139,"snaps":[{"begin":3,"length":2}]},{"pool":145,"snaps":[{"begin":4,"length":2}]},{"pool":147,"snaps":[{"begin":2,"length":2}]},{"pool":152,"snaps":[{"begin":4,"length":4}]},{"pool":196,"snaps":[{"begin":7,"length":1}]},{"pool":200,"snaps":[{"begin":8,"length":1}]},{"pool":202,"snaps":[{"begin":10,"length":1}]},{"pool":203,"snaps":[{"begin":11,"length":2}]},{"pool":206,"snaps":[{"begin":14,"length":1}]},{"pool":249,"snaps":[{"begin":2,"length":1}]},{"pool":250,"snaps":[{"begin":3,"length":1}]},{"pool":253,"snaps":[{"begin":4,"length":1}]},{"pool":257,"snaps":[{"begin":6,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T08:11:11.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 10 librados: watch_flush enter 2024-03-13T08:11:11.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 10 librados: watch_flush exit 2024-03-13T08:11:11.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 20 client.19654.objecter shutdown clearing up homeless session... 2024-03-13T08:11:11.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 10 client.19654.objecter successfully canceled tick 2024-03-13T08:11:11.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 1 -- 172.21.15.132:0/2647985201 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f8be406d560 msgr2=0x7f8be406fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:11.189 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 1 --2- 172.21.15.132:0/2647985201 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f8be406d560 0x7f8be406fa20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8be8006fd0 tx=0x7f8be8008040 comp rx=0 tx=0).stop 2024-03-13T08:11:11.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 10 monclient: shutdown 2024-03-13T08:11:11.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:11.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 1 -- 172.21.15.132:0/2647985201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bf813d670 msgr2=0x7f8bf813dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:11.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 1 --2- 172.21.15.132:0/2647985201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bf813d670 0x7f8bf813dad0 secure :-1 s=READY pgs=4725 cs=0 l=1 rev1=1 crypto rx=0x7f8bec033000 tx=0x7f8bec0046c0 comp rx=0 tx=0).stop 2024-03-13T08:11:11.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 1 -- 172.21.15.132:0/2647985201 shutdown_connections 2024-03-13T08:11:11.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 1 --2- 172.21.15.132:0/2647985201 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f8be406d560 0x7f8be406fa20 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:11.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 1 --2- 172.21.15.132:0/2647985201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bf81dafa0 0x7f8bf81db420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:11.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 1 --2- 172.21.15.132:0/2647985201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bf813d670 0x7f8bf813dad0 unknown :-1 s=CLOSED pgs=4725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:11.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 1 --2- 172.21.15.132:0/2647985201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bf80ad8e0 0x7f8bf813ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:11.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 1 -- 172.21.15.132:0/2647985201 >> 172.21.15.132:0/2647985201 conn(0x7f8bf80b3e20 msgr2=0x7f8bf80b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:11.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 1 -- 172.21.15.132:0/2647985201 shutdown_connections 2024-03-13T08:11:11.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 1 -- 172.21.15.132:0/2647985201 wait complete. 2024-03-13T08:11:11.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:11.188+0000 7f8bfe31c640 1 librados: shutdown 2024-03-13T08:11:12.201 INFO:tasks.ceph:Scrubbing osd.0 2024-03-13T08:11:12.201 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-13T08:11:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.308+0000 7facdd81d640 10 monclient: get_monmap_and_config 2024-03-13T08:11:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.308+0000 7facdd81d640 10 monclient: build_initial_monmap 2024-03-13T08:11:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.308+0000 7facdd81d640 10 monclient: monmap: 2024-03-13T08:11:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:12.308 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:12.309067+0000 2024-03-13T08:11:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:12.309067+0000 2024-03-13T08:11:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:12.309 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:12.309 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.308+0000 7facdd81d640 1 Processor -- start 2024-03-13T08:11:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.308+0000 7facdd81d640 1 -- start start 2024-03-13T08:11:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.308+0000 7facdd81d640 10 monclient: init 2024-03-13T08:11:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.309+0000 7facdd81d640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.309+0000 7facdd81d640 10 monclient: _add_conns ranks=[3,1,2,4,0] 2024-03-13T08:11:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.309+0000 7facdd81d640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7facd80af7c0 0x7facd813d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.309+0000 7facdd81d640 10 monclient(hunting): picked mon.noname-a con 0x7facd80af7c0 addr v2:172.21.15.132:3300/0 2024-03-13T08:11:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.309+0000 7facdd81d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7facd8071700 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.309+0000 7facdd81d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facd813db40 0x7facd8140d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:12.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.309+0000 7facdd81d640 10 monclient(hunting): picked mon.noname-b con 0x7facd813db40 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.309+0000 7facdd81d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facd8141270 con 0x7facd8071700 2024-03-13T08:11:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.309+0000 7facdd81d640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.309+0000 7facdd81d640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facd8071c70 con 0x7facd80af7c0 2024-03-13T08:11:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.309+0000 7facdd81d640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.309+0000 7facdd81d640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd8072810 con 0x7facd813db40 2024-03-13T08:11:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.309+0000 7facdd81d640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.309+0000 7facdd81d640 10 monclient(hunting): authenticate will time out at 3499.717624s 2024-03-13T08:11:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.310+0000 7facd6ffd640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7facd80af7c0 0x7facd813d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.310+0000 7facd77fe640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facd813db40 0x7facd8140d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.310+0000 7facd6ffd640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7facd80af7c0 0x7facd813d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51100/0 (socket says 172.21.15.132:51100) 2024-03-13T08:11:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.310+0000 7facd6ffd640 1 -- 172.21.15.132:0/210904729 learned_addr learned my addr 172.21.15.132:0/210904729 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.310+0000 7facd6ffd640 10 monclient(hunting): get_auth_request con 0x7facd80af7c0 auth_method 0 2024-03-13T08:11:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.310+0000 7facd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.310+0000 7facd6ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.310+0000 7facd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:12.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.310+0000 7facd77fe640 10 monclient(hunting): get_auth_request con 0x7facd813db40 auth_method 0 2024-03-13T08:11:12.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.310+0000 7facd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:12.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.310+0000 7facd77fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:12.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.310+0000 7facd77fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:12.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.310+0000 7facd6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:12.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.310+0000 7facd6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:12.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.310+0000 7facd6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:12.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.311+0000 7facd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:12.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.311+0000 7facd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:12.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.311+0000 7facd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:12.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.312+0000 7facd77fe640 10 monclient(hunting): handle_auth_done global_id 18962 payload 274 2024-03-13T08:11:12.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.312+0000 7facd77fe640 1 -- 172.21.15.132:0/210904729 >> v2:172.21.15.132:3300/0 conn(0x7facd80af7c0 msgr2=0x7facd813d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:12.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.312+0000 7facd77fe640 1 --2- 172.21.15.132:0/210904729 >> v2:172.21.15.132:3300/0 conn(0x7facd80af7c0 0x7facd813d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:12.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.312+0000 7facd77fe640 1 -- 172.21.15.132:0/210904729 >> v1:172.21.15.195:6789/0 conn(0x7facd8071700 legacy=0x7facd8140620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:12.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.312+0000 7facd77fe640 10 monclient: _finish_hunting 0 2024-03-13T08:11:12.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.312+0000 7facd77fe640 1 monclient: found mon.noname-b 2024-03-13T08:11:12.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.312+0000 7facd77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:12.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.312+0000 7facd77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T08:11:12.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.312+0000 7facd77fe640 1 -- 172.21.15.132:0/210904729 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd8072af0 con 0x7facd813db40 2024-03-13T08:11:12.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.314+0000 7facd77fe640 1 --2- 172.21.15.132:0/210904729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facd813db40 0x7facd8140d30 secure :-1 s=READY pgs=3535 cs=0 l=1 rev1=1 crypto rx=0x7facc8009d90 tx=0x7facc802bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33230662b465395f server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:12.317 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.317+0000 7facd5ffb640 1 -- 172.21.15.132:0/210904729 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8039070 con 0x7facd813db40 2024-03-13T08:11:12.318 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.317+0000 7facd5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:12.318 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.317+0000 7facd5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:12.318 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.317+0000 7facd5ffb640 10 monclient: dump: 2024-03-13T08:11:12.318 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:12.318 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:12.318 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:12.318 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:12.318 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:12.318 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:12.318 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:12.318 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:12.318 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:12.318 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:12.318 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.317+0000 7facd5ffb640 10 monclient: _finish_auth 0 2024-03-13T08:11:12.318 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.317+0000 7facd5ffb640 10 monclient: _check_auth_tickets 2024-03-13T08:11:12.319 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.317+0000 7facd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:12.319 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.317+0000 7facd5ffb640 1 -- 172.21.15.132:0/210904729 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc802cb30 con 0x7facd813db40 2024-03-13T08:11:12.319 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.317+0000 7facd5ffb640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:12.319 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.317+0000 7facd5ffb640 1 -- 172.21.15.132:0/210904729 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8034440 con 0x7facd813db40 2024-03-13T08:11:12.319 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.317+0000 7facdd81d640 5 monclient: authenticate success, global_id 18962 2024-03-13T08:11:12.319 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.317+0000 7facdd81d640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:12.319 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.317+0000 7facdd81d640 4 set_mon_vals no callback set 2024-03-13T08:11:12.319 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.317+0000 7facd5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:12.319 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.317+0000 7facd5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T08:11:12.319 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.317+0000 7facd5ffb640 10 monclient: dump: 2024-03-13T08:11:12.319 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:12.319 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:12.319 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:12.319 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:12.319 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:12.320 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:12.320 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:12.320 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:12.320 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:12.320 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:12.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.318+0000 7facdd81d640 10 monclient: shutdown 2024-03-13T08:11:12.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.318+0000 7facdd81d640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:12.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.318+0000 7facdd81d640 1 -- 172.21.15.132:0/210904729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facd813db40 msgr2=0x7facd8140d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:12.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.318+0000 7facdd81d640 1 --2- 172.21.15.132:0/210904729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facd813db40 0x7facd8140d30 secure :-1 s=READY pgs=3535 cs=0 l=1 rev1=1 crypto rx=0x7facc8009d90 tx=0x7facc802bd80 comp rx=0 tx=0).stop 2024-03-13T08:11:12.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.318+0000 7facdd81d640 1 -- 172.21.15.132:0/210904729 shutdown_connections 2024-03-13T08:11:12.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.318+0000 7facdd81d640 1 --2- 172.21.15.132:0/210904729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facd813db40 0x7facd8140d30 unknown :-1 s=CLOSED pgs=3535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:12.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.318+0000 7facdd81d640 1 --2- 172.21.15.132:0/210904729 >> v2:172.21.15.132:3300/0 conn(0x7facd80af7c0 0x7facd813d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:12.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.318+0000 7facdd81d640 1 -- 172.21.15.132:0/210904729 >> 172.21.15.132:0/210904729 conn(0x7facd806acc0 msgr2=0x7facd806b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:12.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.318+0000 7facdd81d640 1 -- 172.21.15.132:0/210904729 shutdown_connections 2024-03-13T08:11:12.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.318+0000 7facdd81d640 1 -- 172.21.15.132:0/210904729 wait complete. 2024-03-13T08:11:12.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.319+0000 7facdd81d640 10 monclient: build_initial_monmap 2024-03-13T08:11:12.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.319+0000 7facdd81d640 10 monclient: monmap: 2024-03-13T08:11:12.324 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:12.324 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:12.324 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:12.324 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:12.324 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:12.324 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:12.324 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:12.324 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:12.324 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:12.324 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:12.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.319+0000 7facdd81d640 1 librados: starting msgr at 2024-03-13T08:11:12.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.319+0000 7facdd81d640 1 librados: starting objecter 2024-03-13T08:11:12.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.319+0000 7facdd81d640 1 Processor -- start 2024-03-13T08:11:12.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 1 -- start start 2024-03-13T08:11:12.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 1 librados: setting wanted keys 2024-03-13T08:11:12.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 1 librados: calling monclient init 2024-03-13T08:11:12.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 10 monclient: init 2024-03-13T08:11:12.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:12.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 10 monclient: _add_conns ranks=[1,0,2] 2024-03-13T08:11:12.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facd8071700 0x7facd81314c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:12.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 10 monclient(hunting): picked mon.noname-b con 0x7facd8071700 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:12.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facd80af7c0 0x7facd806f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:12.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 10 monclient(hunting): picked mon.noname-a con 0x7facd80af7c0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:12.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facd813db40 0x7facd806fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:12.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 10 monclient(hunting): picked mon.noname-c con 0x7facd813db40 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:12.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:12.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd80b0470 con 0x7facd80af7c0 2024-03-13T08:11:12.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:12.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd80b02f0 con 0x7facd8071700 2024-03-13T08:11:12.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:12.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd814acd0 con 0x7facd813db40 2024-03-13T08:11:12.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:12.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.320+0000 7facdd81d640 10 monclient(hunting): authenticate will time out at 3499.728887s 2024-03-13T08:11:12.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.321+0000 7facd77fe640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facd813db40 0x7facd806fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:12.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.321+0000 7facd6ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facd8071700 0x7facd81314c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:12.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.321+0000 7facd77fe640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facd813db40 0x7facd806fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:54700/0 (socket says 172.21.15.132:54700) 2024-03-13T08:11:12.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.321+0000 7facd77fe640 1 -- 172.21.15.132:0/2999696707 learned_addr learned my addr 172.21.15.132:0/2999696707 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:12.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.321+0000 7facd77fe640 10 monclient(hunting): get_auth_request con 0x7facd813db40 auth_method 0 2024-03-13T08:11:12.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.321+0000 7facd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:12.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.321+0000 7facd77fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:12.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.321+0000 7facd77fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:12.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.321+0000 7facd6ffd640 10 monclient(hunting): get_auth_request con 0x7facd8071700 auth_method 0 2024-03-13T08:11:12.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.321+0000 7facd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:12.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.321+0000 7facd6ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:12.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.321+0000 7facd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:12.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.322+0000 7facd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:12.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.322+0000 7facd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:12.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.322+0000 7facd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:12.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.322+0000 7facd6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:12.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.322+0000 7facd6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:12.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.322+0000 7facd6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.324+0000 7facd77fe640 10 monclient(hunting): handle_auth_done global_id 18965 payload 824 2024-03-13T08:11:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.324+0000 7facd77fe640 1 -- 172.21.15.132:0/2999696707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facd8071700 msgr2=0x7facd81314c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:12.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.324+0000 7facd77fe640 1 --2- 172.21.15.132:0/2999696707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facd8071700 0x7facd81314c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.324+0000 7facd77fe640 1 -- 172.21.15.132:0/2999696707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facd80af7c0 msgr2=0x7facd806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.324+0000 7facd77fe640 1 --2- 172.21.15.132:0/2999696707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facd80af7c0 0x7facd806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.324+0000 7facd77fe640 10 monclient: _finish_hunting 0 2024-03-13T08:11:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.324+0000 7facd77fe640 1 monclient: found mon.noname-c 2024-03-13T08:11:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.324+0000 7facd77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.324+0000 7facd77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T08:11:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.324+0000 7facd77fe640 1 -- 172.21.15.132:0/2999696707 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd81db540 con 0x7facd813db40 2024-03-13T08:11:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.327+0000 7facd77fe640 1 --2- 172.21.15.132:0/2999696707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facd813db40 0x7facd806fcd0 secure :-1 s=READY pgs=3536 cs=0 l=1 rev1=1 crypto rx=0x7facc8009d90 tx=0x7facc8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.327+0000 7facbffff640 10 client.?.objecter ms_handle_connect 0x7facd813db40 2024-03-13T08:11:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.327+0000 7facbffff640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facbffff640 1 -- 172.21.15.132:0/2999696707 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8039050 con 0x7facd813db40 2024-03-13T08:11:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facbffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facbffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T08:11:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facbffff640 10 monclient: dump: 2024-03-13T08:11:12.333 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:12.334 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facbffff640 10 monclient: _finish_auth 0 2024-03-13T08:11:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facbffff640 10 monclient: _check_auth_tickets 2024-03-13T08:11:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facbffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facdd81d640 5 monclient: authenticate success, global_id 18965 2024-03-13T08:11:12.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facdd81d640 10 monclient: _renew_subs 2024-03-13T08:11:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facdd81d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facdd81d640 1 -- 172.21.15.132:0/2999696707 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd813ff80 con 0x7facd813db40 2024-03-13T08:11:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facdd81d640 10 client.18965.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facdd81d640 10 monclient: _renew_subs 2024-03-13T08:11:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facdd81d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facdd81d640 1 -- 172.21.15.132:0/2999696707 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd813f5a0 con 0x7facd813db40 2024-03-13T08:11:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facdd81d640 1 librados: init done 2024-03-13T08:11:12.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facdd81d640 10 client.18965.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facdd81d640 15 client.18965.objecter _session_command_op_assign -1 1 2024-03-13T08:11:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facdd81d640 20 client.18965.objecter _get_session osd=-1 returning homeless 2024-03-13T08:11:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facdd81d640 10 client.18965.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facdd81d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:12.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.329+0000 7facdd81d640 1 -- 172.21.15.132:0/2999696707 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7facd806cdf0 con 0x7facd813db40 2024-03-13T08:11:12.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.350+0000 7facbffff640 1 -- 172.21.15.132:0/2999696707 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd806cdf0 con 0x7facd813db40 2024-03-13T08:11:12.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.350+0000 7facbffff640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:12.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.350+0000 7facbffff640 1 -- 172.21.15.132:0/2999696707 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8009450 con 0x7facd813db40 2024-03-13T08:11:12.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.350+0000 7facbffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:12.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.350+0000 7facbffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T08:11:12.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.350+0000 7facbffff640 10 monclient: dump: 2024-03-13T08:11:12.357 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:12.357 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:12.357 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:12.359 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:12.359 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:12.359 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:12.359 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:12.359 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:12.359 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:12.359 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:12.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.350+0000 7facd57fa640 4 set_mon_vals no callback set 2024-03-13T08:11:12.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.351+0000 7facbffff640 1 -- 172.21.15.132:0/2999696707 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7facc803e8d0 con 0x7facd813db40 2024-03-13T08:11:12.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.352+0000 7facbffff640 1 --2- 172.21.15.132:0/2999696707 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7facc406d560 0x7facc406fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:12.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.352+0000 7facd6ffd640 1 --2- 172.21.15.132:0/2999696707 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7facc406d560 0x7facc406fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:12.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.352+0000 7facd6ffd640 10 monclient: get_auth_request con 0x7facc406d560 auth_method 0 2024-03-13T08:11:12.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.352+0000 7facbffff640 10 client.18965.objecter ms_handle_connect 0x7facc406d560 2024-03-13T08:11:12.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.352+0000 7facd6ffd640 1 --2- 172.21.15.132:0/2999696707 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7facc406d560 0x7facc406fa20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7facd0009bd0 tx=0x7facd0009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:12.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 1 -- 172.21.15.132:0/2999696707 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7facc807a640 con 0x7facd813db40 2024-03-13T08:11:12.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 10 client.18965.objecter ms_dispatch 0x7facd8000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:12.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 3 client.18965.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:12.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 3 client.18965.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:12.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 10 client.18965.objecter checking command 1 2024-03-13T08:11:12.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 1 --2- 172.21.15.132:0/2999696707 >> [v2:172.21.15.132:6818/590522438,v1:172.21.15.132:6819/590522438] conn(0x7facc40751d0 0x7facc4077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:12.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 20 client.18965.objecter _get_session s=0x7facc4074b00 osd=0 3 2024-03-13T08:11:12.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 20 client.18965.objecter put_session s=0x7facc4074b00 osd=0 3 2024-03-13T08:11:12.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 15 client.18965.objecter _session_command_op_remove -1 1 2024-03-13T08:11:12.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 20 client.18965.objecter _get_session s=0x7facc4074b00 osd=0 3 2024-03-13T08:11:12.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 20 client.18965.objecter get_session s=0x7facc4074b00 osd=0 3 2024-03-13T08:11:12.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 15 client.18965.objecter _session_command_op_assign 0 1 2024-03-13T08:11:12.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 20 client.18965.objecter put_session s=0x7facc4074b00 osd=0 4 2024-03-13T08:11:12.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 10 client.18965.objecter _send_command 1 2024-03-13T08:11:12.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 1 -- 172.21.15.132:0/2999696707 --> [v2:172.21.15.132:6818/590522438,v1:172.21.15.132:6819/590522438] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7facc4077d40 con 0x7facc40751d0 2024-03-13T08:11:12.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 20 client.18965.objecter dump_active .. 0 homeless 2024-03-13T08:11:12.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 1 -- 172.21.15.132:0/2999696707 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_get_version_reply(handle=1 version=1293) v2 ==== 24+0+0 (secure 0 0 0) 0x7facc8078840 con 0x7facd813db40 2024-03-13T08:11:12.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.355+0000 7facbffff640 10 monclient: handle_get_version_reply finishing 1 version 1293 2024-03-13T08:11:12.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.356+0000 7facd67fc640 1 --2- 172.21.15.132:0/2999696707 >> [v2:172.21.15.132:6818/590522438,v1:172.21.15.132:6819/590522438] conn(0x7facc40751d0 0x7facc4077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:12.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.356+0000 7facd67fc640 10 monclient: get_auth_request con 0x7facc40751d0 auth_method 0 2024-03-13T08:11:12.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.358+0000 7facbffff640 10 client.18965.objecter ms_handle_connect 0x7facc40751d0 2024-03-13T08:11:12.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.358+0000 7facd67fc640 1 --2- 172.21.15.132:0/2999696707 >> [v2:172.21.15.132:6818/590522438,v1:172.21.15.132:6819/590522438] conn(0x7facc40751d0 0x7facc4077670 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:12.382 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.382+0000 7facbffff640 1 -- 172.21.15.132:0/2999696707 <== osd.0 v2:172.21.15.132:6818/590522438 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7facc4077d40 con 0x7facc40751d0 2024-03-13T08:11:12.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.382+0000 7facbffff640 10 client.18965.objecter ms_dispatch 0x7facd8000b90 command_reply(tid 1: 0 ) v1 2024-03-13T08:11:12.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.382+0000 7facbffff640 10 client.18965.objecter _finish_command 1 = system:0 2024-03-13T08:11:12.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.382+0000 7facbffff640 20 client.18965.objecter put_session s=0x7facc4074b00 osd=0 4 2024-03-13T08:11:12.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.382+0000 7facbffff640 15 client.18965.objecter _session_command_op_remove 0 1 2024-03-13T08:11:12.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.395+0000 7facdd81d640 10 client.18965.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-03-13T08:11:12.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.395+0000 7facdd81d640 15 client.18965.objecter _session_command_op_assign -1 2 2024-03-13T08:11:12.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.395+0000 7facdd81d640 20 client.18965.objecter _get_session s=0x7facc4074b00 osd=0 3 2024-03-13T08:11:12.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.395+0000 7facdd81d640 20 client.18965.objecter put_session s=0x7facc4074b00 osd=0 3 2024-03-13T08:11:12.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.395+0000 7facdd81d640 20 client.18965.objecter _get_session s=0x7facc4074b00 osd=0 3 2024-03-13T08:11:12.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.395+0000 7facdd81d640 15 client.18965.objecter _session_command_op_remove -1 2 2024-03-13T08:11:12.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.395+0000 7facdd81d640 20 client.18965.objecter get_session s=0x7facc4074b00 osd=0 3 2024-03-13T08:11:12.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.395+0000 7facdd81d640 15 client.18965.objecter _session_command_op_assign 0 2 2024-03-13T08:11:12.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.395+0000 7facdd81d640 20 client.18965.objecter put_session s=0x7facc4074b00 osd=0 4 2024-03-13T08:11:12.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.395+0000 7facdd81d640 10 client.18965.objecter _send_command 2 2024-03-13T08:11:12.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.395+0000 7facdd81d640 1 -- 172.21.15.132:0/2999696707 --> [v2:172.21.15.132:6818/590522438,v1:172.21.15.132:6819/590522438] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7facd81dbf50 con 0x7facc40751d0 2024-03-13T08:11:12.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.407+0000 7facbffff640 1 -- 172.21.15.132:0/2999696707 <== osd.0 v2:172.21.15.132:6818/590522438 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7facd81dbf50 con 0x7facc40751d0 2024-03-13T08:11:12.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.407+0000 7facbffff640 10 client.18965.objecter ms_dispatch 0x7facd8000b90 command_reply(tid 2: 0 ) v1 2024-03-13T08:11:12.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.407+0000 7facbffff640 10 client.18965.objecter _finish_command 2 = system:0 2024-03-13T08:11:12.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.407+0000 7facbffff640 20 client.18965.objecter put_session s=0x7facc4074b00 osd=0 4 2024-03-13T08:11:12.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.407+0000 7facbffff640 15 client.18965.objecter _session_command_op_remove 0 2 2024-03-13T08:11:12.410 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-13T08:11:12.410 INFO:teuthology.orchestra.run.smithi132.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-13T08:11:12.410 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-13T08:11:12.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 10 librados: watch_flush enter 2024-03-13T08:11:12.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 10 librados: watch_flush exit 2024-03-13T08:11:12.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 10 client.18965.objecter close_session for osd.0 2024-03-13T08:11:12.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 1 -- 172.21.15.132:0/2999696707 >> [v2:172.21.15.132:6818/590522438,v1:172.21.15.132:6819/590522438] conn(0x7facc40751d0 msgr2=0x7facc4077670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:12.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 1 --2- 172.21.15.132:0/2999696707 >> [v2:172.21.15.132:6818/590522438,v1:172.21.15.132:6819/590522438] conn(0x7facc40751d0 0x7facc4077670 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:12.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 20 client.18965.objecter put_session s=0x7facc4074b00 osd=0 1 2024-03-13T08:11:12.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 20 client.18965.objecter shutdown clearing up homeless session... 2024-03-13T08:11:12.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 10 client.18965.objecter successfully canceled tick 2024-03-13T08:11:12.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 1 -- 172.21.15.132:0/2999696707 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7facc406d560 msgr2=0x7facc406fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:12.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 1 --2- 172.21.15.132:0/2999696707 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7facc406d560 0x7facc406fa20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7facd0009bd0 tx=0x7facd0009240 comp rx=0 tx=0).stop 2024-03-13T08:11:12.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 10 monclient: shutdown 2024-03-13T08:11:12.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:12.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 1 -- 172.21.15.132:0/2999696707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facd813db40 msgr2=0x7facd806fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:12.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 1 --2- 172.21.15.132:0/2999696707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facd813db40 0x7facd806fcd0 secure :-1 s=READY pgs=3536 cs=0 l=1 rev1=1 crypto rx=0x7facc8009d90 tx=0x7facc8004a40 comp rx=0 tx=0).stop 2024-03-13T08:11:12.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facd77fe640 1 -- 172.21.15.132:0/2999696707 reap_dead start 2024-03-13T08:11:12.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 1 -- 172.21.15.132:0/2999696707 shutdown_connections 2024-03-13T08:11:12.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 1 -- 172.21.15.132:0/2999696707 >> 172.21.15.132:0/2999696707 conn(0x7facd806acc0 msgr2=0x7facd80b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:12.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 1 -- 172.21.15.132:0/2999696707 shutdown_connections 2024-03-13T08:11:12.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 1 -- 172.21.15.132:0/2999696707 wait complete. 2024-03-13T08:11:12.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.408+0000 7facdd81d640 1 librados: shutdown 2024-03-13T08:11:12.420 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-13T08:11:12.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.520+0000 7fe77f14a640 10 monclient: get_monmap_and_config 2024-03-13T08:11:12.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.520+0000 7fe77f14a640 10 monclient: build_initial_monmap 2024-03-13T08:11:12.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.520+0000 7fe77f14a640 10 monclient: monmap: 2024-03-13T08:11:12.520 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:12.520 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:12.520 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:12.520951+0000 2024-03-13T08:11:12.521 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:12.520951+0000 2024-03-13T08:11:12.521 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:12.521 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:12.521 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:12.521 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:12.521 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:12.521 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:12.521 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:12.521 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:12.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.520+0000 7fe77f14a640 1 Processor -- start 2024-03-13T08:11:12.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.521+0000 7fe77f14a640 1 -- start start 2024-03-13T08:11:12.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.521+0000 7fe77f14a640 10 monclient: init 2024-03-13T08:11:12.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.521+0000 7fe77f14a640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:12.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.521+0000 7fe77f14a640 10 monclient: _add_conns ranks=[0,2,4,1,3] 2024-03-13T08:11:12.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.521+0000 7fe77f14a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe7780b5320 addr v1:172.21.15.132:6789/0 2024-03-13T08:11:12.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.521+0000 7fe77f14a640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7780b0220 0x7fe77813f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:12.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.521+0000 7fe77f14a640 10 monclient(hunting): picked mon.noname-b con 0x7fe7780b0220 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:12.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.521+0000 7fe77f14a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe77813fb70 0x7fe77814cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:12.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.521+0000 7fe77f14a640 10 monclient(hunting): picked mon.noname-c con 0x7fe77813fb70 addr v2:172.21.15.195:3300/0 2024-03-13T08:11:12.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.521+0000 7fe77f14a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7780b1150 con 0x7fe7780b5320 2024-03-13T08:11:12.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.521+0000 7fe77f14a640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:12.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.521+0000 7fe77f14a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7780b0660 con 0x7fe77813fb70 2024-03-13T08:11:12.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.521+0000 7fe77f14a640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:12.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.521+0000 7fe77f14a640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780b13c0 con 0x7fe7780b0220 2024-03-13T08:11:12.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.521+0000 7fe77f14a640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:12.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.521+0000 7fe77f14a640 10 monclient(hunting): authenticate will time out at 3499.929741s 2024-03-13T08:11:12.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.522+0000 7fe777fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7780b0220 0x7fe77813f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:12.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.522+0000 7fe77d6e7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe77813fb70 0x7fe77814cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:12.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.522+0000 7fe77cee6640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fe7780b5320 0x7fe7780b1ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:60794/0 (socket says 172.21.15.132:60794) 2024-03-13T08:11:12.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.522+0000 7fe77cee6640 1 -- 172.21.15.132:0/2453575019 learned_addr learned my addr 172.21.15.132:0/2453575019 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T08:11:12.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.522+0000 7fe777fff640 10 monclient(hunting): get_auth_request con 0x7fe7780b0220 auth_method 0 2024-03-13T08:11:12.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.522+0000 7fe777fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:12.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.522+0000 7fe777fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:12.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.522+0000 7fe777fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:12.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.522+0000 7fe77d6e7640 10 monclient(hunting): get_auth_request con 0x7fe77813fb70 auth_method 0 2024-03-13T08:11:12.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.522+0000 7fe77d6e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:12.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.522+0000 7fe77d6e7640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:12.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.522+0000 7fe77d6e7640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:12.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.523+0000 7fe777fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:12.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.523+0000 7fe777fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:12.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.523+0000 7fe777fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:12.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.523+0000 7fe77d6e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:12.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.523+0000 7fe77d6e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:12.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.523+0000 7fe77d6e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:12.526 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.525+0000 7fe777fff640 10 monclient(hunting): handle_auth_done global_id 18968 payload 274 2024-03-13T08:11:12.526 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.525+0000 7fe777fff640 1 -- 172.21.15.132:0/2453575019 >> v2:172.21.15.195:3300/0 conn(0x7fe77813fb70 msgr2=0x7fe77814cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:12.526 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.525+0000 7fe777fff640 1 --2- 172.21.15.132:0/2453575019 >> v2:172.21.15.195:3300/0 conn(0x7fe77813fb70 0x7fe77814cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:12.526 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.525+0000 7fe777fff640 1 -- 172.21.15.132:0/2453575019 >> v1:172.21.15.132:6789/0 conn(0x7fe7780b5320 legacy=0x7fe7780b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:12.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.525+0000 7fe777fff640 10 monclient: _finish_hunting 0 2024-03-13T08:11:12.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.525+0000 7fe777fff640 1 monclient: found mon.noname-b 2024-03-13T08:11:12.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.525+0000 7fe777fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:12.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.525+0000 7fe777fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T08:11:12.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.525+0000 7fe777fff640 1 -- 172.21.15.132:0/2453575019 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7780b16a0 con 0x7fe7780b0220 2024-03-13T08:11:12.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.525+0000 7fe77d6e7640 1 --2- 172.21.15.132:0/2453575019 >> v2:172.21.15.195:3300/0 conn(0x7fe77813fb70 0x7fe77814cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-13T08:11:12.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.527+0000 7fe777fff640 1 --2- 172.21.15.132:0/2453575019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7780b0220 0x7fe77813f630 secure :-1 s=READY pgs=3537 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0097a0 tx=0x7fe76c02b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e9fccdb0007a625 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:12.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe7777fe640 1 -- 172.21.15.132:0/2453575019 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c038070 con 0x7fe7780b0220 2024-03-13T08:11:12.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe7777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:12.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe7777fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:12.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe7777fe640 10 monclient: dump: 2024-03-13T08:11:12.531 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:12.531 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:12.531 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:12.531 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:12.531 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:12.531 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:12.531 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:12.531 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:12.532 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:12.532 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:12.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe7777fe640 10 monclient: _finish_auth 0 2024-03-13T08:11:12.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe7777fe640 10 monclient: _check_auth_tickets 2024-03-13T08:11:12.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe7777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:12.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe7777fe640 1 -- 172.21.15.132:0/2453575019 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76c031700 con 0x7fe7780b0220 2024-03-13T08:11:12.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe77f14a640 5 monclient: authenticate success, global_id 18968 2024-03-13T08:11:12.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe77f14a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T08:11:12.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe7777fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:12.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe77f14a640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:12.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe7777fe640 1 -- 172.21.15.132:0/2453575019 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c031ad0 con 0x7fe7780b0220 2024-03-13T08:11:12.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe7777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:12.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe77f14a640 4 set_mon_vals no callback set 2024-03-13T08:11:12.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe7777fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T08:11:12.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.530+0000 7fe7777fe640 10 monclient: dump: 2024-03-13T08:11:12.533 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:12.533 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:12.533 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:12.533 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:12.533 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:12.534 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:12.534 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:12.534 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:12.534 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:12.534 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:12.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.531+0000 7fe77f14a640 10 monclient: shutdown 2024-03-13T08:11:12.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.531+0000 7fe77f14a640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:12.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.531+0000 7fe77f14a640 1 -- 172.21.15.132:0/2453575019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7780b0220 msgr2=0x7fe77813f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:12.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.531+0000 7fe77f14a640 1 --2- 172.21.15.132:0/2453575019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7780b0220 0x7fe77813f630 secure :-1 s=READY pgs=3537 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0097a0 tx=0x7fe76c02b7a0 comp rx=0 tx=0).stop 2024-03-13T08:11:12.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.531+0000 7fe77f14a640 1 -- 172.21.15.132:0/2453575019 shutdown_connections 2024-03-13T08:11:12.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.531+0000 7fe77f14a640 1 --2- 172.21.15.132:0/2453575019 >> v2:172.21.15.195:3300/0 conn(0x7fe77813fb70 0x7fe77814cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:12.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.531+0000 7fe77f14a640 1 --2- 172.21.15.132:0/2453575019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7780b0220 0x7fe77813f630 unknown :-1 s=CLOSED pgs=3537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:12.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.531+0000 7fe77f14a640 1 -- 172.21.15.132:0/2453575019 >> 172.21.15.132:0/2453575019 conn(0x7fe77806d2b0 msgr2=0x7fe77806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:12.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.531+0000 7fe77f14a640 1 -- 172.21.15.132:0/2453575019 shutdown_connections 2024-03-13T08:11:12.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.531+0000 7fe77f14a640 1 -- 172.21.15.132:0/2453575019 wait complete. 2024-03-13T08:11:12.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.532+0000 7fe77f14a640 10 monclient: build_initial_monmap 2024-03-13T08:11:12.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.532+0000 7fe77f14a640 10 monclient: monmap: 2024-03-13T08:11:12.540 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:12.540 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:12.540 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:12.540 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:12.540 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:12.540 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:12.540 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:12.540 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:12.540 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:12.540 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:12.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.532+0000 7fe77f14a640 1 librados: starting msgr at 2024-03-13T08:11:12.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.532+0000 7fe77f14a640 1 librados: starting objecter 2024-03-13T08:11:12.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.532+0000 7fe77f14a640 1 Processor -- start 2024-03-13T08:11:12.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.532+0000 7fe77f14a640 1 -- start start 2024-03-13T08:11:12.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.532+0000 7fe77f14a640 1 librados: setting wanted keys 2024-03-13T08:11:12.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.532+0000 7fe77f14a640 1 librados: calling monclient init 2024-03-13T08:11:12.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.532+0000 7fe77f14a640 10 monclient: init 2024-03-13T08:11:12.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77f14a640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:12.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77f14a640 10 monclient: _add_conns ranks=[1,0,2] 2024-03-13T08:11:12.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77f14a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7780b0220 0x7fe7781cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:12.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77f14a640 10 monclient(hunting): picked mon.noname-b con 0x7fe7780b0220 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:12.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77f14a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7780b5320 0x7fe7781d8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:12.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77f14a640 10 monclient(hunting): picked mon.noname-a con 0x7fe7780b5320 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:12.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77f14a640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77813fb70 0x7fe7781d8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:12.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77f14a640 10 monclient(hunting): picked mon.noname-c con 0x7fe77813fb70 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:12.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77f14a640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:12.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77f14a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780b4800 con 0x7fe7780b5320 2024-03-13T08:11:12.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77f14a640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:12.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77f14a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780afd00 con 0x7fe7780b0220 2024-03-13T08:11:12.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77f14a640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:12.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77f14a640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780b4980 con 0x7fe77813fb70 2024-03-13T08:11:12.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77f14a640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:12.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77f14a640 10 monclient(hunting): authenticate will time out at 3499.941607s 2024-03-13T08:11:12.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.533+0000 7fe77d6e7640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77813fb70 0x7fe7781d8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:12.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.534+0000 7fe777fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7780b5320 0x7fe7781d8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:12.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.534+0000 7fe77cee6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7780b0220 0x7fe7781cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:12.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.534+0000 7fe777fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7780b5320 0x7fe7781d8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51128/0 (socket says 172.21.15.132:51128) 2024-03-13T08:11:12.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.534+0000 7fe777fff640 1 -- 172.21.15.132:0/2539840223 learned_addr learned my addr 172.21.15.132:0/2539840223 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:12.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.534+0000 7fe777fff640 10 monclient(hunting): get_auth_request con 0x7fe7780b5320 auth_method 0 2024-03-13T08:11:12.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.534+0000 7fe777fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:12.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.534+0000 7fe777fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:12.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.534+0000 7fe777fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:12.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.534+0000 7fe77d6e7640 10 monclient(hunting): get_auth_request con 0x7fe77813fb70 auth_method 0 2024-03-13T08:11:12.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.534+0000 7fe77d6e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:12.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.534+0000 7fe77d6e7640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:12.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.534+0000 7fe77d6e7640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:12.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.534+0000 7fe77cee6640 10 monclient(hunting): get_auth_request con 0x7fe7780b0220 auth_method 0 2024-03-13T08:11:12.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.534+0000 7fe77cee6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:12.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.534+0000 7fe77cee6640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:12.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.534+0000 7fe77cee6640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:12.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.535+0000 7fe77cee6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:12.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.535+0000 7fe77cee6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:12.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.535+0000 7fe77cee6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:12.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.535+0000 7fe777fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:12.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.535+0000 7fe777fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:12.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.535+0000 7fe777fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:12.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.535+0000 7fe77d6e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:12.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.535+0000 7fe77d6e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:12.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.535+0000 7fe77d6e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:12.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.537+0000 7fe77cee6640 10 monclient(hunting): handle_auth_done global_id 19663 payload 824 2024-03-13T08:11:12.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.537+0000 7fe77cee6640 1 -- 172.21.15.132:0/2539840223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77813fb70 msgr2=0x7fe7781d8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:12.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.537+0000 7fe77cee6640 1 --2- 172.21.15.132:0/2539840223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77813fb70 0x7fe7781d8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:12.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.537+0000 7fe77cee6640 1 -- 172.21.15.132:0/2539840223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7780b5320 msgr2=0x7fe7781d8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:12.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.537+0000 7fe77cee6640 1 --2- 172.21.15.132:0/2539840223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7780b5320 0x7fe7781d8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:12.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.537+0000 7fe77cee6640 10 monclient: _finish_hunting 0 2024-03-13T08:11:12.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.537+0000 7fe77cee6640 1 monclient: found mon.noname-b 2024-03-13T08:11:12.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.537+0000 7fe77cee6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:12.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.537+0000 7fe77cee6640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T08:11:12.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.537+0000 7fe77cee6640 1 -- 172.21.15.132:0/2539840223 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7781cd9f0 con 0x7fe7780b0220 2024-03-13T08:11:12.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.540+0000 7fe77cee6640 1 --2- 172.21.15.132:0/2539840223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7780b0220 0x7fe7781cbb60 secure :-1 s=READY pgs=4726 cs=0 l=1 rev1=1 crypto rx=0x7fe77000ca80 tx=0x7fe77000ce80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:12.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.540+0000 7fe7757fa640 10 client.?.objecter ms_handle_connect 0x7fe7780b0220 2024-03-13T08:11:12.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.540+0000 7fe7757fa640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:12.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe7757fa640 1 -- 172.21.15.132:0/2539840223 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770017070 con 0x7fe7780b0220 2024-03-13T08:11:12.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe7757fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:12.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe7757fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:12.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe7757fa640 10 monclient: dump: 2024-03-13T08:11:12.547 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:12.547 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:12.547 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:12.547 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:12.547 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:12.547 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:12.547 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:12.547 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:12.548 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:12.548 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:12.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe7757fa640 10 monclient: _finish_auth 0 2024-03-13T08:11:12.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe7757fa640 10 monclient: _check_auth_tickets 2024-03-13T08:11:12.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe7757fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:12.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe7757fa640 1 -- 172.21.15.132:0/2539840223 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77000ab70 con 0x7fe7780b0220 2024-03-13T08:11:12.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe77f14a640 5 monclient: authenticate success, global_id 19663 2024-03-13T08:11:12.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe77f14a640 10 monclient: _renew_subs 2024-03-13T08:11:12.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe77f14a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:12.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe77f14a640 1 -- 172.21.15.132:0/2539840223 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7781cdcd0 con 0x7fe7780b0220 2024-03-13T08:11:12.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe7757fa640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:12.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe7757fa640 1 -- 172.21.15.132:0/2539840223 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770012400 con 0x7fe7780b0220 2024-03-13T08:11:12.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe7757fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:12.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe7757fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:12.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe7757fa640 10 monclient: dump: 2024-03-13T08:11:12.549 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:12.549 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:12.549 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:12.549 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:12.549 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:12.549 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:12.549 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:12.549 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:12.550 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:12.550 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:12.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe77f14a640 10 client.19663.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:12.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe776ffd640 4 set_mon_vals no callback set 2024-03-13T08:11:12.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe77f14a640 10 monclient: _renew_subs 2024-03-13T08:11:12.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe77f14a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:12.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe77f14a640 1 -- 172.21.15.132:0/2539840223 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7781ce0f0 con 0x7fe7780b0220 2024-03-13T08:11:12.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.543+0000 7fe77f14a640 1 librados: init done 2024-03-13T08:11:12.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.544+0000 7fe77f14a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:12.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.544+0000 7fe77f14a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:12.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.544+0000 7fe77f14a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:12.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.544+0000 7fe77f14a640 1 -- 172.21.15.132:0/2539840223 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe77804f000 con 0x7fe7780b0220 2024-03-13T08:11:12.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.798+0000 7fe7757fa640 1 -- 172.21.15.132:0/2539840223 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fe77001a070 con 0x7fe7780b0220 2024-03-13T08:11:12.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.798+0000 7fe7757fa640 1 --2- 172.21.15.132:0/2539840223 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fe76406d560 0x7fe76406fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:12.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.798+0000 7fe777fff640 1 --2- 172.21.15.132:0/2539840223 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fe76406d560 0x7fe76406fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:12.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.798+0000 7fe777fff640 10 monclient: get_auth_request con 0x7fe76406d560 auth_method 0 2024-03-13T08:11:12.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.799+0000 7fe7757fa640 10 client.19663.objecter ms_handle_connect 0x7fe76406d560 2024-03-13T08:11:12.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.799+0000 7fe777fff640 1 --2- 172.21.15.132:0/2539840223 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fe76406d560 0x7fe76406fa20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe76c02b7e0 tx=0x7fe76c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:12.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.800+0000 7fe7757fa640 1 -- 172.21.15.132:0/2539840223 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7fe7700572e0 con 0x7fe7780b0220 2024-03-13T08:11:12.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.800+0000 7fe7757fa640 10 client.19663.objecter ms_dispatch 0x7fe778000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:12.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.800+0000 7fe7757fa640 3 client.19663.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:12.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.800+0000 7fe7757fa640 3 client.19663.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:12.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.800+0000 7fe7757fa640 20 client.19663.objecter dump_active .. 0 homeless 2024-03-13T08:11:12.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.811+0000 7fe7757fa640 1 -- 172.21.15.132:0/2539840223 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fe770021070 con 0x7fe7780b0220 2024-03-13T08:11:12.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.811+0000 7fe7757fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:12.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:12.811+0000 7fe7757fa640 10 monclient: _finish_command 1 = system:0 2024-03-13T08:11:13.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.010+0000 7fe77f14a640 1 -- 172.21.15.132:0/2539840223 --> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7781d7e30 con 0x7fe76406d560 2024-03-13T08:11:13.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.011+0000 7fe7757fa640 1 -- 172.21.15.132:0/2539840223 <== mgr.4104 v2:172.21.15.132:6800/3847550149 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe76c031820 con 0x7fe76406d560 2024-03-13T08:11:13.011 INFO:teuthology.orchestra.run.smithi132.stderr:instructed osd(s) 0 to deep-scrub 2024-03-13T08:11:13.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 10 librados: watch_flush enter 2024-03-13T08:11:13.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 10 librados: watch_flush exit 2024-03-13T08:11:13.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 20 client.19663.objecter shutdown clearing up homeless session... 2024-03-13T08:11:13.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 10 client.19663.objecter successfully canceled tick 2024-03-13T08:11:13.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 1 -- 172.21.15.132:0/2539840223 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fe76406d560 msgr2=0x7fe76406fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:13.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 1 --2- 172.21.15.132:0/2539840223 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fe76406d560 0x7fe76406fa20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe76c02b7e0 tx=0x7fe76c008030 comp rx=0 tx=0).stop 2024-03-13T08:11:13.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 10 monclient: shutdown 2024-03-13T08:11:13.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:13.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 1 -- 172.21.15.132:0/2539840223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7780b0220 msgr2=0x7fe7781cbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:13.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 1 --2- 172.21.15.132:0/2539840223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7780b0220 0x7fe7781cbb60 secure :-1 s=READY pgs=4726 cs=0 l=1 rev1=1 crypto rx=0x7fe77000ca80 tx=0x7fe77000ce80 comp rx=0 tx=0).stop 2024-03-13T08:11:13.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 1 -- 172.21.15.132:0/2539840223 shutdown_connections 2024-03-13T08:11:13.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 1 --2- 172.21.15.132:0/2539840223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77813fb70 0x7fe7781d8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 1 --2- 172.21.15.132:0/2539840223 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fe76406d560 0x7fe76406fa20 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 1 --2- 172.21.15.132:0/2539840223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7780b5320 0x7fe7781d8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 1 --2- 172.21.15.132:0/2539840223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7780b0220 0x7fe7781cbb60 unknown :-1 s=CLOSED pgs=4726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 1 -- 172.21.15.132:0/2539840223 >> 172.21.15.132:0/2539840223 conn(0x7fe77806d2b0 msgr2=0x7fe7780b6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:13.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 1 -- 172.21.15.132:0/2539840223 shutdown_connections 2024-03-13T08:11:13.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 1 -- 172.21.15.132:0/2539840223 wait complete. 2024-03-13T08:11:13.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.013+0000 7fe77f14a640 1 librados: shutdown 2024-03-13T08:11:13.028 INFO:tasks.ceph:Scrubbing osd.1 2024-03-13T08:11:13.028 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-13T08:11:13.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.132+0000 7fca90ec0640 10 monclient: get_monmap_and_config 2024-03-13T08:11:13.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.132+0000 7fca90ec0640 10 monclient: build_initial_monmap 2024-03-13T08:11:13.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.132+0000 7fca90ec0640 10 monclient: monmap: 2024-03-13T08:11:13.132 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:13.133 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:13.133 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:13.133215+0000 2024-03-13T08:11:13.133 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:13.133215+0000 2024-03-13T08:11:13.133 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:13.133 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:13.133 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:13.133 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:13.133 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:13.133 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:13.133 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:13.133 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:13.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 1 Processor -- start 2024-03-13T08:11:13.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 1 -- start start 2024-03-13T08:11:13.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 10 monclient: init 2024-03-13T08:11:13.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:13.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 10 monclient: _add_conns ranks=[2,4,0,1,3] 2024-03-13T08:11:13.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c0b8760 0x7fca8c1426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:13.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 10 monclient(hunting): picked mon.noname-b con 0x7fca8c0b8760 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:13.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fca8c0b8b40 0x7fca8c0b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:13.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 10 monclient(hunting): picked mon.noname-c con 0x7fca8c0b8b40 addr v2:172.21.15.195:3300/0 2024-03-13T08:11:13.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fca8c0b24e0 addr v1:172.21.15.132:6789/0 2024-03-13T08:11:13.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8c0b1b00 con 0x7fca8c0b24e0 2024-03-13T08:11:13.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:13.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8c0b1400 con 0x7fca8c0b8b40 2024-03-13T08:11:13.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:13.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c0b16e0 con 0x7fca8c0b8760 2024-03-13T08:11:13.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:13.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.133+0000 7fca90ec0640 10 monclient(hunting): authenticate will time out at 3500.541925s 2024-03-13T08:11:13.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.134+0000 7fca8ad9d640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fca8c0b24e0 0x7fca8c0aea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:60802/0 (socket says 172.21.15.132:60802) 2024-03-13T08:11:13.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.134+0000 7fca8ad9d640 1 -- 172.21.15.132:0/2372088624 learned_addr learned my addr 172.21.15.132:0/2372088624 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T08:11:13.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.134+0000 7fca89d9b640 1 --2- 172.21.15.132:0/2372088624 >> v2:172.21.15.195:3300/0 conn(0x7fca8c0b8b40 0x7fca8c0b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:13.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.134+0000 7fca8a59c640 1 --2- 172.21.15.132:0/2372088624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c0b8760 0x7fca8c1426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:13.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.134+0000 7fca89d9b640 10 monclient(hunting): get_auth_request con 0x7fca8c0b8b40 auth_method 0 2024-03-13T08:11:13.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.134+0000 7fca89d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:13.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.134+0000 7fca89d9b640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:13.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.134+0000 7fca89d9b640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:13.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.134+0000 7fca8a59c640 10 monclient(hunting): get_auth_request con 0x7fca8c0b8760 auth_method 0 2024-03-13T08:11:13.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.134+0000 7fca8a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:13.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.134+0000 7fca8a59c640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:13.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.134+0000 7fca8a59c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:13.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.135+0000 7fca89d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:13.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.135+0000 7fca89d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:13.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.135+0000 7fca89d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:13.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.135+0000 7fca8a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:13.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.135+0000 7fca8a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:13.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.135+0000 7fca8a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:13.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.137+0000 7fca89d9b640 10 monclient(hunting): handle_auth_done global_id 19666 payload 274 2024-03-13T08:11:13.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.137+0000 7fca89d9b640 1 -- 172.21.15.132:0/2372088624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c0b8760 msgr2=0x7fca8c1426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:13.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.137+0000 7fca89d9b640 1 --2- 172.21.15.132:0/2372088624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c0b8760 0x7fca8c1426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.137+0000 7fca89d9b640 1 -- 172.21.15.132:0/2372088624 >> v1:172.21.15.132:6789/0 conn(0x7fca8c0b24e0 legacy=0x7fca8c0aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:13.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.137+0000 7fca89d9b640 10 monclient: _finish_hunting 0 2024-03-13T08:11:13.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.137+0000 7fca89d9b640 1 monclient: found mon.noname-c 2024-03-13T08:11:13.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.137+0000 7fca89d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:13.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.137+0000 7fca89d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T08:11:13.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.137+0000 7fca89d9b640 1 -- 172.21.15.132:0/2372088624 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c0b4840 con 0x7fca8c0b8b40 2024-03-13T08:11:13.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.139+0000 7fca89d9b640 1 --2- 172.21.15.132:0/2372088624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c0b8b40 0x7fca8c0b1fa0 secure :-1 s=READY pgs=4727 cs=0 l=1 rev1=1 crypto rx=0x7fca800097a0 tx=0x7fca8002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce79f30b16a24676 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:13.144 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.143+0000 7fca8959a640 1 -- 172.21.15.132:0/2372088624 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca800315a0 con 0x7fca8c0b8b40 2024-03-13T08:11:13.144 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.143+0000 7fca8959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:13.144 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.143+0000 7fca8959a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T08:11:13.144 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.143+0000 7fca8959a640 10 monclient: dump: 2024-03-13T08:11:13.144 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:13.144 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:13.144 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:13.144 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:13.144 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:13.144 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:13.144 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:13.144 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:13.145 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:13.145 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:13.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.143+0000 7fca8959a640 10 monclient: _finish_auth 0 2024-03-13T08:11:13.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.143+0000 7fca8959a640 10 monclient: _check_auth_tickets 2024-03-13T08:11:13.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.143+0000 7fca8959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:13.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.143+0000 7fca8959a640 1 -- 172.21.15.132:0/2372088624 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca80037070 con 0x7fca8c0b8b40 2024-03-13T08:11:13.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.143+0000 7fca8959a640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:13.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.143+0000 7fca8959a640 1 -- 172.21.15.132:0/2372088624 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80031960 con 0x7fca8c0b8b40 2024-03-13T08:11:13.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.143+0000 7fca90ec0640 5 monclient: authenticate success, global_id 19666 2024-03-13T08:11:13.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.143+0000 7fca90ec0640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:13.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.143+0000 7fca90ec0640 4 set_mon_vals no callback set 2024-03-13T08:11:13.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.143+0000 7fca8959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:13.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.143+0000 7fca8959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:13.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.143+0000 7fca8959a640 10 monclient: dump: 2024-03-13T08:11:13.145 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:13.146 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:13.146 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:13.146 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:13.146 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:13.146 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:13.146 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:13.146 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:13.146 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:13.146 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:13.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.144+0000 7fca90ec0640 10 monclient: shutdown 2024-03-13T08:11:13.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.144+0000 7fca90ec0640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:13.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.144+0000 7fca90ec0640 1 -- 172.21.15.132:0/2372088624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c0b8b40 msgr2=0x7fca8c0b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:13.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.144+0000 7fca90ec0640 1 --2- 172.21.15.132:0/2372088624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c0b8b40 0x7fca8c0b1fa0 secure :-1 s=READY pgs=4727 cs=0 l=1 rev1=1 crypto rx=0x7fca800097a0 tx=0x7fca8002b840 comp rx=0 tx=0).stop 2024-03-13T08:11:13.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.144+0000 7fca90ec0640 1 -- 172.21.15.132:0/2372088624 shutdown_connections 2024-03-13T08:11:13.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.144+0000 7fca90ec0640 1 --2- 172.21.15.132:0/2372088624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c0b8b40 0x7fca8c0b1fa0 unknown :-1 s=CLOSED pgs=4727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.144+0000 7fca90ec0640 1 --2- 172.21.15.132:0/2372088624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c0b8760 0x7fca8c1426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.144+0000 7fca90ec0640 1 -- 172.21.15.132:0/2372088624 >> 172.21.15.132:0/2372088624 conn(0x7fca8c06d4f0 msgr2=0x7fca8c0b8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:13.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.144+0000 7fca90ec0640 1 -- 172.21.15.132:0/2372088624 shutdown_connections 2024-03-13T08:11:13.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.144+0000 7fca90ec0640 1 -- 172.21.15.132:0/2372088624 wait complete. 2024-03-13T08:11:13.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.145+0000 7fca90ec0640 10 monclient: build_initial_monmap 2024-03-13T08:11:13.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.145+0000 7fca90ec0640 10 monclient: monmap: 2024-03-13T08:11:13.150 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:13.150 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:13.153 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:13.154 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:13.154 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:13.154 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:13.154 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:13.154 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:13.154 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:13.154 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:13.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.145+0000 7fca90ec0640 1 librados: starting msgr at 2024-03-13T08:11:13.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.145+0000 7fca90ec0640 1 librados: starting objecter 2024-03-13T08:11:13.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.145+0000 7fca90ec0640 1 Processor -- start 2024-03-13T08:11:13.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.145+0000 7fca90ec0640 1 -- start start 2024-03-13T08:11:13.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.145+0000 7fca90ec0640 1 librados: setting wanted keys 2024-03-13T08:11:13.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.145+0000 7fca90ec0640 1 librados: calling monclient init 2024-03-13T08:11:13.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.145+0000 7fca90ec0640 10 monclient: init 2024-03-13T08:11:13.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca90ec0640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:13.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca90ec0640 10 monclient: _add_conns ranks=[0,1,2] 2024-03-13T08:11:13.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca90ec0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca8c0b8760 0x7fca8c13e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:13.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca90ec0640 10 monclient(hunting): picked mon.noname-a con 0x7fca8c0b8760 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:13.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca90ec0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c0ad740 0x7fca8c0aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:13.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca90ec0640 10 monclient(hunting): picked mon.noname-b con 0x7fca8c0ad740 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:13.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca90ec0640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c0ab9d0 0x7fca8c0abe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:13.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca90ec0640 10 monclient(hunting): picked mon.noname-c con 0x7fca8c0ab9d0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:13.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca90ec0640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:13.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca90ec0640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c0b00c0 con 0x7fca8c0b8760 2024-03-13T08:11:13.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca90ec0640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:13.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca90ec0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c0aff40 con 0x7fca8c0ad740 2024-03-13T08:11:13.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca90ec0640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:13.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca90ec0640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c0b0240 con 0x7fca8c0ab9d0 2024-03-13T08:11:13.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca90ec0640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:13.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca90ec0640 10 monclient(hunting): authenticate will time out at 3500.554463s 2024-03-13T08:11:13.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca8a59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca8c0b8760 0x7fca8c13e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:13.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca8ad9d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c0ab9d0 0x7fca8c0abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:13.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.146+0000 7fca89d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c0ad740 0x7fca8c0aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:13.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca8a59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca8c0b8760 0x7fca8c13e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51134/0 (socket says 172.21.15.132:51134) 2024-03-13T08:11:13.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca8a59c640 1 -- 172.21.15.132:0/1457310747 learned_addr learned my addr 172.21.15.132:0/1457310747 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:13.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca8a59c640 10 monclient(hunting): get_auth_request con 0x7fca8c0b8760 auth_method 0 2024-03-13T08:11:13.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca8a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:13.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca8a59c640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:13.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca8a59c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:13.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca89d9b640 10 monclient(hunting): get_auth_request con 0x7fca8c0ad740 auth_method 0 2024-03-13T08:11:13.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca89d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:13.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca89d9b640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:13.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca89d9b640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:13.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca8ad9d640 10 monclient(hunting): get_auth_request con 0x7fca8c0ab9d0 auth_method 0 2024-03-13T08:11:13.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca8ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:13.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca8ad9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:13.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca8ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:13.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca8a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:13.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca8a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:13.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca8a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:13.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca8ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:13.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca8ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:13.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca8ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:13.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca89d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:13.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca89d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:13.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.147+0000 7fca89d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:13.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.150+0000 7fca89d9b640 10 monclient(hunting): handle_auth_done global_id 19669 payload 824 2024-03-13T08:11:13.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.150+0000 7fca89d9b640 1 -- 172.21.15.132:0/1457310747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c0ab9d0 msgr2=0x7fca8c0abe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:13.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.150+0000 7fca89d9b640 1 --2- 172.21.15.132:0/1457310747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c0ab9d0 0x7fca8c0abe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.150+0000 7fca89d9b640 1 -- 172.21.15.132:0/1457310747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca8c0b8760 msgr2=0x7fca8c13e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:13.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.150+0000 7fca89d9b640 1 --2- 172.21.15.132:0/1457310747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca8c0b8760 0x7fca8c13e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.150+0000 7fca89d9b640 10 monclient: _finish_hunting 0 2024-03-13T08:11:13.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.150+0000 7fca89d9b640 1 monclient: found mon.noname-b 2024-03-13T08:11:13.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.150+0000 7fca89d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:13.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.150+0000 7fca89d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T08:11:13.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.150+0000 7fca89d9b640 1 -- 172.21.15.132:0/1457310747 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c0ad040 con 0x7fca8c0ad740 2024-03-13T08:11:13.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.150+0000 7fca8a59c640 1 --2- 172.21.15.132:0/1457310747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca8c0b8760 0x7fca8c13e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-13T08:11:13.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.152+0000 7fca89d9b640 1 --2- 172.21.15.132:0/1457310747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c0ad740 0x7fca8c0aacc0 secure :-1 s=READY pgs=4728 cs=0 l=1 rev1=1 crypto rx=0x7fca8c069d80 tx=0x7fca800046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:13.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.152+0000 7fca7affd640 10 client.?.objecter ms_handle_connect 0x7fca8c0ad740 2024-03-13T08:11:13.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.152+0000 7fca7affd640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:13.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca7affd640 1 -- 172.21.15.132:0/1457310747 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80032630 con 0x7fca8c0ad740 2024-03-13T08:11:13.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca7affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:13.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca7affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:13.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca7affd640 10 monclient: dump: 2024-03-13T08:11:13.161 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:13.161 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:13.161 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:13.161 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:13.162 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:13.162 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:13.162 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:13.162 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:13.162 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:13.162 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:13.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca7affd640 10 monclient: _finish_auth 0 2024-03-13T08:11:13.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca7affd640 10 monclient: _check_auth_tickets 2024-03-13T08:11:13.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca7affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:13.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca7affd640 1 -- 172.21.15.132:0/1457310747 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca800327f0 con 0x7fca8c0ad740 2024-03-13T08:11:13.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca90ec0640 5 monclient: authenticate success, global_id 19669 2024-03-13T08:11:13.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca90ec0640 10 monclient: _renew_subs 2024-03-13T08:11:13.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca90ec0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:13.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca90ec0640 1 -- 172.21.15.132:0/1457310747 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8c0ab200 con 0x7fca8c0ad740 2024-03-13T08:11:13.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca7affd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:13.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca7affd640 1 -- 172.21.15.132:0/1457310747 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80032be0 con 0x7fca8c0ad740 2024-03-13T08:11:13.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca7affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:13.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca7affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:13.163 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca7affd640 10 monclient: dump: 2024-03-13T08:11:13.163 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:13.163 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:13.163 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:13.164 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:13.164 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:13.164 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:13.164 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:13.164 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:13.164 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:13.164 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:13.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca88d99640 4 set_mon_vals no callback set 2024-03-13T08:11:13.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca90ec0640 10 client.19669.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:13.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca90ec0640 10 monclient: _renew_subs 2024-03-13T08:11:13.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca90ec0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:13.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca90ec0640 1 -- 172.21.15.132:0/1457310747 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c0ab500 con 0x7fca8c0ad740 2024-03-13T08:11:13.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.156+0000 7fca90ec0640 1 librados: init done 2024-03-13T08:11:13.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.157+0000 7fca90ec0640 10 client.19669.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:13.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.157+0000 7fca90ec0640 15 client.19669.objecter _session_command_op_assign -1 1 2024-03-13T08:11:13.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.157+0000 7fca90ec0640 20 client.19669.objecter _get_session osd=-1 returning homeless 2024-03-13T08:11:13.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.157+0000 7fca90ec0640 10 client.19669.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:13.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.157+0000 7fca90ec0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:13.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.157+0000 7fca90ec0640 1 -- 172.21.15.132:0/1457310747 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca8c04f000 con 0x7fca8c0ad740 2024-03-13T08:11:13.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.176+0000 7fca7affd640 1 -- 172.21.15.132:0/1457310747 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fca8004c410 con 0x7fca8c0ad740 2024-03-13T08:11:13.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.176+0000 7fca7affd640 1 --2- 172.21.15.132:0/1457310747 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fca6c06d590 0x7fca6c06fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:13.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.176+0000 7fca8a59c640 1 --2- 172.21.15.132:0/1457310747 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fca6c06d590 0x7fca6c06fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:13.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.176+0000 7fca8a59c640 10 monclient: get_auth_request con 0x7fca6c06d590 auth_method 0 2024-03-13T08:11:13.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.176+0000 7fca7affd640 10 client.19669.objecter ms_handle_connect 0x7fca6c06d590 2024-03-13T08:11:13.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.176+0000 7fca8a59c640 1 --2- 172.21.15.132:0/1457310747 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fca6c06d590 0x7fca6c06fa50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fca84009a40 tx=0x7fca8400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:13.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.178+0000 7fca7affd640 1 -- 172.21.15.132:0/1457310747 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7fca8002f030 con 0x7fca8c0ad740 2024-03-13T08:11:13.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.178+0000 7fca7affd640 10 client.19669.objecter ms_dispatch 0x7fca8c000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:13.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.178+0000 7fca7affd640 3 client.19669.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:13.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.178+0000 7fca7affd640 3 client.19669.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:13.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.178+0000 7fca7affd640 10 client.19669.objecter checking command 1 2024-03-13T08:11:13.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.178+0000 7fca7affd640 1 --2- 172.21.15.132:0/1457310747 >> [v2:172.21.15.132:6802/1071068206,v1:172.21.15.132:6803/1071068206] conn(0x7fca6c075200 0x7fca6c0776a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:13.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.178+0000 7fca7affd640 20 client.19669.objecter _get_session s=0x7fca6c074b30 osd=1 3 2024-03-13T08:11:13.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.178+0000 7fca7affd640 20 client.19669.objecter put_session s=0x7fca6c074b30 osd=1 3 2024-03-13T08:11:13.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.178+0000 7fca7affd640 15 client.19669.objecter _session_command_op_remove -1 1 2024-03-13T08:11:13.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.178+0000 7fca7affd640 20 client.19669.objecter _get_session s=0x7fca6c074b30 osd=1 3 2024-03-13T08:11:13.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.178+0000 7fca7affd640 20 client.19669.objecter get_session s=0x7fca6c074b30 osd=1 3 2024-03-13T08:11:13.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.178+0000 7fca7affd640 15 client.19669.objecter _session_command_op_assign 1 1 2024-03-13T08:11:13.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.178+0000 7fca7affd640 20 client.19669.objecter put_session s=0x7fca6c074b30 osd=1 4 2024-03-13T08:11:13.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.178+0000 7fca7affd640 10 client.19669.objecter _send_command 1 2024-03-13T08:11:13.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.178+0000 7fca7affd640 1 -- 172.21.15.132:0/1457310747 --> [v2:172.21.15.132:6802/1071068206,v1:172.21.15.132:6803/1071068206] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca80043ec0 con 0x7fca6c075200 2024-03-13T08:11:13.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.178+0000 7fca7affd640 20 client.19669.objecter dump_active .. 0 homeless 2024-03-13T08:11:13.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.179+0000 7fca8ad9d640 1 --2- 172.21.15.132:0/1457310747 >> [v2:172.21.15.132:6802/1071068206,v1:172.21.15.132:6803/1071068206] conn(0x7fca6c075200 0x7fca6c0776a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:13.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.179+0000 7fca8ad9d640 10 monclient: get_auth_request con 0x7fca6c075200 auth_method 0 2024-03-13T08:11:13.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.179+0000 7fca7affd640 1 -- 172.21.15.132:0/1457310747 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=1293) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca80076c90 con 0x7fca8c0ad740 2024-03-13T08:11:13.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.179+0000 7fca7affd640 10 monclient: handle_get_version_reply finishing 1 version 1293 2024-03-13T08:11:13.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.182+0000 7fca8ad9d640 1 --2- 172.21.15.132:0/1457310747 >> [v2:172.21.15.132:6802/1071068206,v1:172.21.15.132:6803/1071068206] conn(0x7fca6c075200 0x7fca6c0776a0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:13.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.182+0000 7fca7affd640 10 client.19669.objecter ms_handle_connect 0x7fca6c075200 2024-03-13T08:11:13.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.205+0000 7fca7affd640 1 -- 172.21.15.132:0/1457310747 <== osd.1 v2:172.21.15.132:6802/1071068206 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fca80043ec0 con 0x7fca6c075200 2024-03-13T08:11:13.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.205+0000 7fca7affd640 10 client.19669.objecter ms_dispatch 0x7fca8c000b90 command_reply(tid 1: 0 ) v1 2024-03-13T08:11:13.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.205+0000 7fca7affd640 10 client.19669.objecter _finish_command 1 = system:0 2024-03-13T08:11:13.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.205+0000 7fca7affd640 20 client.19669.objecter put_session s=0x7fca6c074b30 osd=1 4 2024-03-13T08:11:13.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.205+0000 7fca7affd640 15 client.19669.objecter _session_command_op_remove 1 1 2024-03-13T08:11:13.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.218+0000 7fca90ec0640 10 client.19669.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-03-13T08:11:13.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.218+0000 7fca90ec0640 15 client.19669.objecter _session_command_op_assign -1 2 2024-03-13T08:11:13.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.218+0000 7fca90ec0640 20 client.19669.objecter _get_session s=0x7fca6c074b30 osd=1 3 2024-03-13T08:11:13.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.218+0000 7fca90ec0640 20 client.19669.objecter put_session s=0x7fca6c074b30 osd=1 3 2024-03-13T08:11:13.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.218+0000 7fca90ec0640 20 client.19669.objecter _get_session s=0x7fca6c074b30 osd=1 3 2024-03-13T08:11:13.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.218+0000 7fca90ec0640 15 client.19669.objecter _session_command_op_remove -1 2 2024-03-13T08:11:13.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.218+0000 7fca90ec0640 20 client.19669.objecter get_session s=0x7fca6c074b30 osd=1 3 2024-03-13T08:11:13.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.218+0000 7fca90ec0640 15 client.19669.objecter _session_command_op_assign 1 2 2024-03-13T08:11:13.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.218+0000 7fca90ec0640 20 client.19669.objecter put_session s=0x7fca6c074b30 osd=1 4 2024-03-13T08:11:13.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.218+0000 7fca90ec0640 10 client.19669.objecter _send_command 2 2024-03-13T08:11:13.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.218+0000 7fca90ec0640 1 -- 172.21.15.132:0/1457310747 --> [v2:172.21.15.132:6802/1071068206,v1:172.21.15.132:6803/1071068206] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fca8c141bd0 con 0x7fca6c075200 2024-03-13T08:11:13.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.233+0000 7fca7affd640 1 -- 172.21.15.132:0/1457310747 <== osd.1 v2:172.21.15.132:6802/1071068206 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fca8c141bd0 con 0x7fca6c075200 2024-03-13T08:11:13.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.233+0000 7fca7affd640 10 client.19669.objecter ms_dispatch 0x7fca8c000b90 command_reply(tid 2: 0 ) v1 2024-03-13T08:11:13.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.233+0000 7fca7affd640 10 client.19669.objecter _finish_command 2 = system:0 2024-03-13T08:11:13.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.233+0000 7fca7affd640 20 client.19669.objecter put_session s=0x7fca6c074b30 osd=1 4 2024-03-13T08:11:13.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.233+0000 7fca7affd640 15 client.19669.objecter _session_command_op_remove 1 2 2024-03-13T08:11:13.234 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-13T08:11:13.234 INFO:teuthology.orchestra.run.smithi132.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-13T08:11:13.234 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-13T08:11:13.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 10 librados: watch_flush enter 2024-03-13T08:11:13.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 10 librados: watch_flush exit 2024-03-13T08:11:13.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 10 client.19669.objecter close_session for osd.1 2024-03-13T08:11:13.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 1 -- 172.21.15.132:0/1457310747 >> [v2:172.21.15.132:6802/1071068206,v1:172.21.15.132:6803/1071068206] conn(0x7fca6c075200 msgr2=0x7fca6c0776a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:13.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 1 --2- 172.21.15.132:0/1457310747 >> [v2:172.21.15.132:6802/1071068206,v1:172.21.15.132:6803/1071068206] conn(0x7fca6c075200 0x7fca6c0776a0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 20 client.19669.objecter put_session s=0x7fca6c074b30 osd=1 1 2024-03-13T08:11:13.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 20 client.19669.objecter shutdown clearing up homeless session... 2024-03-13T08:11:13.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 10 client.19669.objecter successfully canceled tick 2024-03-13T08:11:13.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 1 -- 172.21.15.132:0/1457310747 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fca6c06d590 msgr2=0x7fca6c06fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:13.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 1 --2- 172.21.15.132:0/1457310747 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fca6c06d590 0x7fca6c06fa50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fca84009a40 tx=0x7fca8400b040 comp rx=0 tx=0).stop 2024-03-13T08:11:13.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 10 monclient: shutdown 2024-03-13T08:11:13.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:13.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 1 -- 172.21.15.132:0/1457310747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c0ad740 msgr2=0x7fca8c0aacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:13.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 1 --2- 172.21.15.132:0/1457310747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c0ad740 0x7fca8c0aacc0 secure :-1 s=READY pgs=4728 cs=0 l=1 rev1=1 crypto rx=0x7fca8c069d80 tx=0x7fca800046c0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca8ad9d640 1 -- 172.21.15.132:0/1457310747 reap_dead start 2024-03-13T08:11:13.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 1 -- 172.21.15.132:0/1457310747 shutdown_connections 2024-03-13T08:11:13.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 1 -- 172.21.15.132:0/1457310747 >> 172.21.15.132:0/1457310747 conn(0x7fca8c06d4f0 msgr2=0x7fca8c13aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:13.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 1 -- 172.21.15.132:0/1457310747 shutdown_connections 2024-03-13T08:11:13.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 1 -- 172.21.15.132:0/1457310747 wait complete. 2024-03-13T08:11:13.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.234+0000 7fca90ec0640 1 librados: shutdown 2024-03-13T08:11:13.245 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-13T08:11:13.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.346+0000 7f43b5406640 10 monclient: get_monmap_and_config 2024-03-13T08:11:13.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.346+0000 7f43b5406640 10 monclient: build_initial_monmap 2024-03-13T08:11:13.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.346+0000 7f43b5406640 10 monclient: monmap: 2024-03-13T08:11:13.347 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:13.347 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:13.347 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:13.347175+0000 2024-03-13T08:11:13.348 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:13.347175+0000 2024-03-13T08:11:13.348 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:13.348 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:13.348 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:13.348 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:13.348 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:13.348 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:13.348 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:13.348 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:13.348 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.347+0000 7f43b5406640 1 Processor -- start 2024-03-13T08:11:13.348 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.347+0000 7f43b5406640 1 -- start start 2024-03-13T08:11:13.348 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.347+0000 7f43b5406640 10 monclient: init 2024-03-13T08:11:13.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.347+0000 7f43b5406640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:13.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.348+0000 7f43b5406640 10 monclient: _add_conns ranks=[1,4,2,0,3] 2024-03-13T08:11:13.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.348+0000 7f43b5406640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f43b00b5320 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:13.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.348+0000 7f43b5406640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f43b00b0220 0x7f43b013f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:13.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.348+0000 7f43b5406640 10 monclient(hunting): picked mon.noname-c con 0x7f43b00b0220 addr v2:172.21.15.195:3300/0 2024-03-13T08:11:13.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.348+0000 7f43b5406640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43b013fb70 0x7f43b014cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:13.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.348+0000 7f43b5406640 10 monclient(hunting): picked mon.noname-b con 0x7f43b013fb70 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:13.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.348+0000 7f43b5406640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b00b1150 con 0x7f43b00b5320 2024-03-13T08:11:13.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.348+0000 7f43b5406640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:13.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.348+0000 7f43b5406640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b00b0690 con 0x7f43b00b0220 2024-03-13T08:11:13.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.348+0000 7f43b5406640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:13.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.348+0000 7f43b5406640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b013ffa0 con 0x7f43b013fb70 2024-03-13T08:11:13.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.348+0000 7f43b5406640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:13.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.348+0000 7f43b5406640 10 monclient(hunting): authenticate will time out at 3500.756180s 2024-03-13T08:11:13.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.348+0000 7f43af7fe640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43b013fb70 0x7f43b014cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:13.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.348+0000 7f43ae7fc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f43b00b0220 0x7f43b013f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:13.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.349+0000 7f43ae7fc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f43b00b0220 0x7f43b013f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:48986/0 (socket says 172.21.15.132:48986) 2024-03-13T08:11:13.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.349+0000 7f43ae7fc640 1 -- 172.21.15.132:0/258622647 learned_addr learned my addr 172.21.15.132:0/258622647 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:13.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.349+0000 7f43ae7fc640 10 monclient(hunting): get_auth_request con 0x7f43b00b0220 auth_method 0 2024-03-13T08:11:13.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.349+0000 7f43ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:13.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.349+0000 7f43ae7fc640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:13.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.349+0000 7f43ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:13.352 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.349+0000 7f43af7fe640 10 monclient(hunting): get_auth_request con 0x7f43b013fb70 auth_method 0 2024-03-13T08:11:13.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.349+0000 7f43af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:13.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.349+0000 7f43af7fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:13.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.349+0000 7f43af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:13.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.349+0000 7f43ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:13.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.349+0000 7f43ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:13.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.349+0000 7f43ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:13.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.349+0000 7f43af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:13.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.349+0000 7f43af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:13.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.349+0000 7f43af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:13.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.351+0000 7f43ae7fc640 10 monclient(hunting): handle_auth_done global_id 19672 payload 274 2024-03-13T08:11:13.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.351+0000 7f43ae7fc640 1 -- 172.21.15.132:0/258622647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43b013fb70 msgr2=0x7f43b014cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:13.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.351+0000 7f43ae7fc640 1 --2- 172.21.15.132:0/258622647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43b013fb70 0x7f43b014cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.351+0000 7f43ae7fc640 1 -- 172.21.15.132:0/258622647 >> v1:172.21.15.195:6789/0 conn(0x7f43b00b5320 legacy=0x7f43b00b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:13.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.351+0000 7f43ae7fc640 10 monclient: _finish_hunting 0 2024-03-13T08:11:13.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.351+0000 7f43ae7fc640 1 monclient: found mon.noname-c 2024-03-13T08:11:13.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.351+0000 7f43ae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:13.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.351+0000 7f43ae7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T08:11:13.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.351+0000 7f43ae7fc640 1 -- 172.21.15.132:0/258622647 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b00b13c0 con 0x7f43b00b0220 2024-03-13T08:11:13.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.354+0000 7f43ae7fc640 1 --2- 172.21.15.132:0/258622647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43b00b0220 0x7f43b013f630 secure :-1 s=READY pgs=4729 cs=0 l=1 rev1=1 crypto rx=0x7f43b006ddf0 tx=0x7f43a402b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2145a95a43cca771 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:13.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.358+0000 7f43adffb640 1 -- 172.21.15.132:0/258622647 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4038070 con 0x7f43b00b0220 2024-03-13T08:11:13.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.358+0000 7f43adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:13.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.358+0000 7f43adffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T08:11:13.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.358+0000 7f43adffb640 10 monclient: dump: 2024-03-13T08:11:13.362 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:13.362 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:13.362 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:13.362 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:13.362 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:13.362 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:13.362 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:13.363 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:13.363 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:13.363 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:13.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.358+0000 7f43adffb640 10 monclient: _finish_auth 0 2024-03-13T08:11:13.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.358+0000 7f43adffb640 10 monclient: _check_auth_tickets 2024-03-13T08:11:13.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.358+0000 7f43adffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:13.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.358+0000 7f43adffb640 1 -- 172.21.15.132:0/258622647 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a4031520 con 0x7f43b00b0220 2024-03-13T08:11:13.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.358+0000 7f43adffb640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:13.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.358+0000 7f43adffb640 1 -- 172.21.15.132:0/258622647 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a40318f0 con 0x7f43b00b0220 2024-03-13T08:11:13.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.358+0000 7f43b5406640 5 monclient: authenticate success, global_id 19672 2024-03-13T08:11:13.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.358+0000 7f43b5406640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:13.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.358+0000 7f43b5406640 4 set_mon_vals no callback set 2024-03-13T08:11:13.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.358+0000 7f43adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:13.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.358+0000 7f43adffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:13.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.358+0000 7f43adffb640 10 monclient: dump: 2024-03-13T08:11:13.364 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:13.364 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:13.364 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:13.365 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:13.365 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:13.365 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:13.365 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:13.365 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:13.365 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:13.365 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:13.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.359+0000 7f43b5406640 10 monclient: shutdown 2024-03-13T08:11:13.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.359+0000 7f43b5406640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:13.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.359+0000 7f43b5406640 1 -- 172.21.15.132:0/258622647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43b00b0220 msgr2=0x7f43b013f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:13.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.359+0000 7f43b5406640 1 --2- 172.21.15.132:0/258622647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43b00b0220 0x7f43b013f630 secure :-1 s=READY pgs=4729 cs=0 l=1 rev1=1 crypto rx=0x7f43b006ddf0 tx=0x7f43a402b7c0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.359+0000 7f43b5406640 1 -- 172.21.15.132:0/258622647 shutdown_connections 2024-03-13T08:11:13.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.359+0000 7f43b5406640 1 --2- 172.21.15.132:0/258622647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43b013fb70 0x7f43b014cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.359+0000 7f43b5406640 1 --2- 172.21.15.132:0/258622647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43b00b0220 0x7f43b013f630 unknown :-1 s=CLOSED pgs=4729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.359+0000 7f43b5406640 1 -- 172.21.15.132:0/258622647 >> 172.21.15.132:0/258622647 conn(0x7f43b006d2b0 msgr2=0x7f43b006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:13.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.359+0000 7f43b5406640 1 -- 172.21.15.132:0/258622647 shutdown_connections 2024-03-13T08:11:13.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.359+0000 7f43b5406640 1 -- 172.21.15.132:0/258622647 wait complete. 2024-03-13T08:11:13.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.359+0000 7f43b5406640 10 monclient: build_initial_monmap 2024-03-13T08:11:13.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.359+0000 7f43b5406640 10 monclient: monmap: 2024-03-13T08:11:13.366 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:13.367 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:13.367 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:13.367 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:13.367 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:13.367 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:13.367 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:13.367 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:13.367 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:13.367 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:13.370 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 1 librados: starting msgr at 2024-03-13T08:11:13.370 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 1 librados: starting objecter 2024-03-13T08:11:13.370 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 1 Processor -- start 2024-03-13T08:11:13.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 1 -- start start 2024-03-13T08:11:13.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 1 librados: setting wanted keys 2024-03-13T08:11:13.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 1 librados: calling monclient init 2024-03-13T08:11:13.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 10 monclient: init 2024-03-13T08:11:13.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:13.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T08:11:13.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43b00b0220 0x7f43b00ac380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:13.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 10 monclient(hunting): picked mon.noname-a con 0x7f43b00b0220 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:13.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43b00b5320 0x7f43b00a9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:13.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 10 monclient(hunting): picked mon.noname-c con 0x7f43b00b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:13.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43b013fb70 0x7f43b00a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:13.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 10 monclient(hunting): picked mon.noname-b con 0x7f43b013fb70 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:13.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:13.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00b4800 con 0x7f43b00b0220 2024-03-13T08:11:13.371 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:13.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00afd00 con 0x7f43b013fb70 2024-03-13T08:11:13.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:13.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00b4980 con 0x7f43b00b5320 2024-03-13T08:11:13.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:13.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.360+0000 7f43b5406640 10 monclient(hunting): authenticate will time out at 3500.769010s 2024-03-13T08:11:13.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43ae7fc640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43b00b5320 0x7f43b00a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:13.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43aeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43b00b0220 0x7f43b00ac380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:13.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43af7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43b013fb70 0x7f43b00a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:13.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43aeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43b00b0220 0x7f43b00ac380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51148/0 (socket says 172.21.15.132:51148) 2024-03-13T08:11:13.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43aeffd640 1 -- 172.21.15.132:0/1530982850 learned_addr learned my addr 172.21.15.132:0/1530982850 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:13.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43aeffd640 10 monclient(hunting): get_auth_request con 0x7f43b00b0220 auth_method 0 2024-03-13T08:11:13.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:13.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43aeffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:13.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:13.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43af7fe640 10 monclient(hunting): get_auth_request con 0x7f43b013fb70 auth_method 0 2024-03-13T08:11:13.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:13.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43af7fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:13.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:13.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43ae7fc640 10 monclient(hunting): get_auth_request con 0x7f43b00b5320 auth_method 0 2024-03-13T08:11:13.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:13.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43ae7fc640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:13.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.361+0000 7f43ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:13.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.362+0000 7f43af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:13.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.362+0000 7f43af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:13.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.362+0000 7f43af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:13.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.362+0000 7f43aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:13.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.362+0000 7f43aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:13.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.362+0000 7f43aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:13.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.362+0000 7f43ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:13.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.362+0000 7f43ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:13.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.362+0000 7f43ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:13.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.364+0000 7f43af7fe640 10 monclient(hunting): handle_auth_done global_id 19675 payload 824 2024-03-13T08:11:13.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.364+0000 7f43af7fe640 1 -- 172.21.15.132:0/1530982850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43b00b5320 msgr2=0x7f43b00a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:13.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.364+0000 7f43af7fe640 1 --2- 172.21.15.132:0/1530982850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43b00b5320 0x7f43b00a9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.364+0000 7f43af7fe640 1 -- 172.21.15.132:0/1530982850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43b00b0220 msgr2=0x7f43b00ac380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:13.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.364+0000 7f43af7fe640 1 --2- 172.21.15.132:0/1530982850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43b00b0220 0x7f43b00ac380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.364+0000 7f43af7fe640 10 monclient: _finish_hunting 0 2024-03-13T08:11:13.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.364+0000 7f43af7fe640 1 monclient: found mon.noname-b 2024-03-13T08:11:13.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.364+0000 7f43af7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:13.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.364+0000 7f43af7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T08:11:13.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.364+0000 7f43af7fe640 1 -- 172.21.15.132:0/1530982850 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b00ae490 con 0x7f43b013fb70 2024-03-13T08:11:13.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.367+0000 7f43af7fe640 1 --2- 172.21.15.132:0/1530982850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43b013fb70 0x7f43b00a9c70 secure :-1 s=READY pgs=4730 cs=0 l=1 rev1=1 crypto rx=0x7f43a000ebd0 tx=0x7f43a000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:13.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.367+0000 7f43977fe640 10 client.?.objecter ms_handle_connect 0x7f43b013fb70 2024-03-13T08:11:13.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.367+0000 7f43977fe640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:13.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43977fe640 1 -- 172.21.15.132:0/1530982850 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0018020 con 0x7f43b013fb70 2024-03-13T08:11:13.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:13.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43977fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:13.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43977fe640 10 monclient: dump: 2024-03-13T08:11:13.378 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:13.378 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:13.378 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:13.378 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:13.378 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:13.378 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:13.378 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:13.378 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:13.378 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:13.378 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:13.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43977fe640 10 monclient: _finish_auth 0 2024-03-13T08:11:13.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43977fe640 10 monclient: _check_auth_tickets 2024-03-13T08:11:13.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:13.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43977fe640 1 -- 172.21.15.132:0/1530982850 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a000cd20 con 0x7f43b013fb70 2024-03-13T08:11:13.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43b5406640 5 monclient: authenticate success, global_id 19675 2024-03-13T08:11:13.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43b5406640 10 monclient: _renew_subs 2024-03-13T08:11:13.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43b5406640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:13.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43b5406640 1 -- 172.21.15.132:0/1530982850 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b01403e0 con 0x7f43b013fb70 2024-03-13T08:11:13.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43977fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:13.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43977fe640 1 -- 172.21.15.132:0/1530982850 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a00146c0 con 0x7f43b013fb70 2024-03-13T08:11:13.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:13.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43977fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:13.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43977fe640 10 monclient: dump: 2024-03-13T08:11:13.379 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:13.379 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:13.379 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:13.379 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:13.379 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:13.379 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:13.379 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:13.380 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:13.380 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:13.380 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:13.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43ad7fa640 4 set_mon_vals no callback set 2024-03-13T08:11:13.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43b5406640 10 client.19675.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:13.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43b5406640 10 monclient: _renew_subs 2024-03-13T08:11:13.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43b5406640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:13.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43b5406640 1 -- 172.21.15.132:0/1530982850 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b00aa1b0 con 0x7f43b013fb70 2024-03-13T08:11:13.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.371+0000 7f43b5406640 1 librados: init done 2024-03-13T08:11:13.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.372+0000 7f43b5406640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:13.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.372+0000 7f43b5406640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:13.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.372+0000 7f43b5406640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:13.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.372+0000 7f43b5406640 1 -- 172.21.15.132:0/1530982850 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b004f000 con 0x7f43b013fb70 2024-03-13T08:11:13.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.615+0000 7f43977fe640 1 -- 172.21.15.132:0/1530982850 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f43a0005d10 con 0x7f43b013fb70 2024-03-13T08:11:13.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.616+0000 7f43977fe640 1 --2- 172.21.15.132:0/1530982850 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f439c06d5b0 0x7f439c06fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:13.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.616+0000 7f43aeffd640 1 --2- 172.21.15.132:0/1530982850 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f439c06d5b0 0x7f439c06fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:13.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.616+0000 7f43aeffd640 10 monclient: get_auth_request con 0x7f439c06d5b0 auth_method 0 2024-03-13T08:11:13.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.616+0000 7f43977fe640 10 client.19675.objecter ms_handle_connect 0x7f439c06d5b0 2024-03-13T08:11:13.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.616+0000 7f43aeffd640 1 --2- 172.21.15.132:0/1530982850 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f439c06d5b0 0x7f439c06fa70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f43a80018b0 tx=0x7f43a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:13.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.619+0000 7f43977fe640 1 -- 172.21.15.132:0/1530982850 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f43a00580e0 con 0x7f43b013fb70 2024-03-13T08:11:13.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.619+0000 7f43977fe640 10 client.19675.objecter ms_dispatch 0x7f43b0000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:13.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.619+0000 7f43977fe640 3 client.19675.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:13.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.619+0000 7f43977fe640 3 client.19675.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:13.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.619+0000 7f43977fe640 20 client.19675.objecter dump_active .. 0 homeless 2024-03-13T08:11:13.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.629+0000 7f43977fe640 1 -- 172.21.15.132:0/1530982850 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f43a0022030 con 0x7f43b013fb70 2024-03-13T08:11:13.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.629+0000 7f43977fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:13.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.629+0000 7f43977fe640 10 monclient: _finish_command 1 = system:0 2024-03-13T08:11:13.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.812+0000 7f43b5406640 1 -- 172.21.15.132:0/1530982850 --> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f43b0067e10 con 0x7f439c06d5b0 2024-03-13T08:11:13.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.813+0000 7f43977fe640 1 -- 172.21.15.132:0/1530982850 <== mgr.4104 v2:172.21.15.132:6800/3847550149 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f43b0067e10 con 0x7f439c06d5b0 2024-03-13T08:11:13.813 INFO:teuthology.orchestra.run.smithi132.stderr:instructed osd(s) 1 to deep-scrub 2024-03-13T08:11:13.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 10 librados: watch_flush enter 2024-03-13T08:11:13.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 10 librados: watch_flush exit 2024-03-13T08:11:13.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 20 client.19675.objecter shutdown clearing up homeless session... 2024-03-13T08:11:13.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 10 client.19675.objecter successfully canceled tick 2024-03-13T08:11:13.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 1 -- 172.21.15.132:0/1530982850 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f439c06d5b0 msgr2=0x7f439c06fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:13.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 1 --2- 172.21.15.132:0/1530982850 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f439c06d5b0 0x7f439c06fa70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f43a80018b0 tx=0x7f43a8009040 comp rx=0 tx=0).stop 2024-03-13T08:11:13.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 10 monclient: shutdown 2024-03-13T08:11:13.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:13.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 1 -- 172.21.15.132:0/1530982850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43b013fb70 msgr2=0x7f43b00a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:13.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 1 --2- 172.21.15.132:0/1530982850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43b013fb70 0x7f43b00a9c70 secure :-1 s=READY pgs=4730 cs=0 l=1 rev1=1 crypto rx=0x7f43a000ebd0 tx=0x7f43a000c490 comp rx=0 tx=0).stop 2024-03-13T08:11:13.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 1 -- 172.21.15.132:0/1530982850 shutdown_connections 2024-03-13T08:11:13.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 1 --2- 172.21.15.132:0/1530982850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43b013fb70 0x7f43b00a9c70 unknown :-1 s=CLOSED pgs=4730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 1 --2- 172.21.15.132:0/1530982850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43b00b5320 0x7f43b00a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 1 --2- 172.21.15.132:0/1530982850 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f439c06d5b0 0x7f439c06fa70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 1 --2- 172.21.15.132:0/1530982850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43b00b0220 0x7f43b00ac380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:13.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 1 -- 172.21.15.132:0/1530982850 >> 172.21.15.132:0/1530982850 conn(0x7f43b006d2b0 msgr2=0x7f43b013b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:13.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 1 -- 172.21.15.132:0/1530982850 shutdown_connections 2024-03-13T08:11:13.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 1 -- 172.21.15.132:0/1530982850 wait complete. 2024-03-13T08:11:13.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.815+0000 7f43b5406640 1 librados: shutdown 2024-03-13T08:11:13.829 INFO:tasks.ceph:Scrubbing osd.2 2024-03-13T08:11:13.829 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-13T08:11:13.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.935+0000 7f7d0ba4d640 10 monclient: get_monmap_and_config 2024-03-13T08:11:13.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.935+0000 7f7d0ba4d640 10 monclient: build_initial_monmap 2024-03-13T08:11:13.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.935+0000 7f7d0ba4d640 10 monclient: monmap: 2024-03-13T08:11:13.935 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:13.935 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:13.935 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:13.935866+0000 2024-03-13T08:11:13.935 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:13.935866+0000 2024-03-13T08:11:13.935 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:13.935 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:13.936 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:13.936 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:13.936 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:13.936 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:13.936 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:13.936 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:13.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.935+0000 7f7d0ba4d640 1 Processor -- start 2024-03-13T08:11:13.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.935+0000 7f7d0ba4d640 1 -- start start 2024-03-13T08:11:13.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.935+0000 7f7d0ba4d640 10 monclient: init 2024-03-13T08:11:13.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.936+0000 7f7d0ba4d640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:13.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.936+0000 7f7d0ba4d640 10 monclient: _add_conns ranks=[1,3,0,2,4] 2024-03-13T08:11:13.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.936+0000 7f7d0ba4d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7d040b52a0 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:13.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.936+0000 7f7d0ba4d640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7d040b24b0 0x7f7d04145c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:13.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.936+0000 7f7d0ba4d640 10 monclient(hunting): picked mon.noname-a con 0x7f7d040b24b0 addr v2:172.21.15.132:3300/0 2024-03-13T08:11:13.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.936+0000 7f7d0ba4d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7d04140120 addr v1:172.21.15.132:6789/0 2024-03-13T08:11:13.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.936+0000 7f7d0ba4d640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d041468d0 con 0x7f7d04140120 2024-03-13T08:11:13.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.936+0000 7f7d0ba4d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d040b4790 con 0x7f7d040b52a0 2024-03-13T08:11:13.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.936+0000 7f7d0ba4d640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:13.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.936+0000 7f7d0ba4d640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d040b5680 con 0x7f7d040b24b0 2024-03-13T08:11:13.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.936+0000 7f7d0ba4d640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:13.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.936+0000 7f7d0ba4d640 10 monclient(hunting): authenticate will time out at 3501.344678s 2024-03-13T08:11:13.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.937+0000 7f7d08fe8640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7d040b24b0 0x7f7d04145c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:13.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.937+0000 7f7d09fea640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f7d04140120 0x7f7d041461c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:60814/0 (socket says 172.21.15.132:60814) 2024-03-13T08:11:13.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.937+0000 7f7d09fea640 1 -- 172.21.15.132:0/4285251301 learned_addr learned my addr 172.21.15.132:0/4285251301 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T08:11:13.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.939+0000 7f7d08fe8640 10 monclient(hunting): get_auth_request con 0x7f7d040b24b0 auth_method 0 2024-03-13T08:11:13.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.939+0000 7f7d08fe8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:13.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.939+0000 7f7d08fe8640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:13.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.939+0000 7f7d08fe8640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:13.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.944+0000 7f7cfbfff640 1 -- 172.21.15.132:0/4285251301 <== mon.0 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211047101 0 0) 0x7f7d041468d0 con 0x7f7d04140120 2024-03-13T08:11:13.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.944+0000 7f7cfbfff640 10 monclient(hunting): _negotiate 2024-03-13T08:11:13.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.944+0000 7f7cfbfff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:13.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.944+0000 7f7cfbfff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:13.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.944+0000 7f7cfbfff640 10 monclient(hunting): my global_id is 16029 2024-03-13T08:11:13.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.944+0000 7f7cfbfff640 1 -- 172.21.15.132:0/4285251301 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cf0003520 con 0x7f7d04140120 2024-03-13T08:11:13.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.945+0000 7f7d08fe8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:13.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.945+0000 7f7d08fe8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:13.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.945+0000 7f7d08fe8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:13.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.948+0000 7f7d08fe8640 10 monclient(hunting): handle_auth_done global_id 16032 payload 274 2024-03-13T08:11:13.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.948+0000 7f7d08fe8640 1 -- 172.21.15.132:0/4285251301 >> v1:172.21.15.195:6789/0 conn(0x7f7d040b52a0 legacy=0x7f7d0413fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:13.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.948+0000 7f7d08fe8640 1 -- 172.21.15.132:0/4285251301 >> v1:172.21.15.132:6789/0 conn(0x7f7d04140120 legacy=0x7f7d041461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:13.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.948+0000 7f7d08fe8640 10 monclient: _finish_hunting 0 2024-03-13T08:11:13.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.948+0000 7f7d08fe8640 1 monclient: found mon.noname-a 2024-03-13T08:11:13.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.948+0000 7f7d08fe8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:13.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.948+0000 7f7d08fe8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T08:11:13.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.948+0000 7f7d08fe8640 1 -- 172.21.15.132:0/4285251301 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d040a9ef0 con 0x7f7d040b24b0 2024-03-13T08:11:13.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:13.953+0000 7f7d08fe8640 1 --2- 172.21.15.132:0/4285251301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d040b24b0 0x7f7d04145c80 secure :-1 s=READY pgs=3263 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40097a0 tx=0x7f7cf402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d97c473f766e802 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:14.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7cfbfff640 1 -- 172.21.15.132:0/4285251301 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4035020 con 0x7f7d040b24b0 2024-03-13T08:11:14.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7cfbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:14.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7cfbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T08:11:14.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7cfbfff640 10 monclient: dump: 2024-03-13T08:11:14.033 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:14.033 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:14.033 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.033 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.033 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:14.033 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.033 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:14.033 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:14.033 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:14.033 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7cfbfff640 10 monclient: _finish_auth 0 2024-03-13T08:11:14.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7cfbfff640 10 monclient: _check_auth_tickets 2024-03-13T08:11:14.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7cfbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:14.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7cfbfff640 1 -- 172.21.15.132:0/4285251301 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf4031830 con 0x7f7d040b24b0 2024-03-13T08:11:14.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7d0ba4d640 5 monclient: authenticate success, global_id 16032 2024-03-13T08:11:14.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7d0ba4d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T08:11:14.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7cfbfff640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:14.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7cfbfff640 1 -- 172.21.15.132:0/4285251301 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4031c00 con 0x7f7d040b24b0 2024-03-13T08:11:14.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7cfbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:14.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7cfbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T08:11:14.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7cfbfff640 10 monclient: dump: 2024-03-13T08:11:14.035 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:14.035 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:14.035 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.035 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.035 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:14.035 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.035 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:14.035 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:14.035 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:14.035 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7d0ba4d640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:14.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.031+0000 7f7d0ba4d640 4 set_mon_vals no callback set 2024-03-13T08:11:14.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.032+0000 7f7d0ba4d640 10 monclient: shutdown 2024-03-13T08:11:14.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.032+0000 7f7d0ba4d640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:14.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.032+0000 7f7d0ba4d640 1 -- 172.21.15.132:0/4285251301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d040b24b0 msgr2=0x7f7d04145c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.032+0000 7f7d0ba4d640 1 --2- 172.21.15.132:0/4285251301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d040b24b0 0x7f7d04145c80 secure :-1 s=READY pgs=3263 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40097a0 tx=0x7f7cf402b8d0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.032+0000 7f7d0ba4d640 1 -- 172.21.15.132:0/4285251301 shutdown_connections 2024-03-13T08:11:14.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.032+0000 7f7d0ba4d640 1 --2- 172.21.15.132:0/4285251301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d040b24b0 0x7f7d04145c80 unknown :-1 s=CLOSED pgs=3263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.032+0000 7f7d0ba4d640 1 -- 172.21.15.132:0/4285251301 >> 172.21.15.132:0/4285251301 conn(0x7f7d04066e20 msgr2=0x7f7d04067230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:14.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.032+0000 7f7d0ba4d640 1 -- 172.21.15.132:0/4285251301 shutdown_connections 2024-03-13T08:11:14.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.032+0000 7f7d0ba4d640 1 -- 172.21.15.132:0/4285251301 wait complete. 2024-03-13T08:11:14.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.033+0000 7f7d0ba4d640 10 monclient: build_initial_monmap 2024-03-13T08:11:14.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.033+0000 7f7d0ba4d640 10 monclient: monmap: 2024-03-13T08:11:14.040 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:14.040 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:14.040 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:14.040 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:14.040 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:14.040 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.040 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:14.040 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:14.041 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:14.041 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.033+0000 7f7d0ba4d640 1 librados: starting msgr at 2024-03-13T08:11:14.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.033+0000 7f7d0ba4d640 1 librados: starting objecter 2024-03-13T08:11:14.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.033+0000 7f7d0ba4d640 1 Processor -- start 2024-03-13T08:11:14.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.033+0000 7f7d0ba4d640 1 -- start start 2024-03-13T08:11:14.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.033+0000 7f7d0ba4d640 1 librados: setting wanted keys 2024-03-13T08:11:14.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.033+0000 7f7d0ba4d640 1 librados: calling monclient init 2024-03-13T08:11:14.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.033+0000 7f7d0ba4d640 10 monclient: init 2024-03-13T08:11:14.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d0ba4d640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:14.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d0ba4d640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T08:11:14.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d0ba4d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d040b24b0 0x7f7d04131170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d0ba4d640 10 monclient(hunting): picked mon.noname-a con 0x7f7d040b24b0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:14.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d0ba4d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d040b52a0 0x7f7d04144db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d0ba4d640 10 monclient(hunting): picked mon.noname-c con 0x7f7d040b52a0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:14.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d0ba4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d04140120 0x7f7d0413e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d0ba4d640 10 monclient(hunting): picked mon.noname-b con 0x7f7d04140120 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:14.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d0ba4d640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:14.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d0ba4d640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0414e1d0 con 0x7f7d040b24b0 2024-03-13T08:11:14.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d0ba4d640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:14.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d0ba4d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d040aa4b0 con 0x7f7d04140120 2024-03-13T08:11:14.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d0ba4d640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:14.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d0ba4d640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0414e350 con 0x7f7d040b52a0 2024-03-13T08:11:14.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d0ba4d640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:14.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d0ba4d640 10 monclient(hunting): authenticate will time out at 3501.442313s 2024-03-13T08:11:14.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.034+0000 7f7d08fe8640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d040b52a0 0x7f7d04144db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.035+0000 7f7d08fe8640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d040b52a0 0x7f7d04144db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:54750/0 (socket says 172.21.15.132:54750) 2024-03-13T08:11:14.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.035+0000 7f7d08fe8640 1 -- 172.21.15.132:0/2735114572 learned_addr learned my addr 172.21.15.132:0/2735114572 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:14.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.035+0000 7f7d08fe8640 10 monclient(hunting): get_auth_request con 0x7f7d040b52a0 auth_method 0 2024-03-13T08:11:14.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.035+0000 7f7d08fe8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:14.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.035+0000 7f7d08fe8640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:14.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.035+0000 7f7d08fe8640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:14.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.035+0000 7f7d08fe8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:14.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.035+0000 7f7d08fe8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:14.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.035+0000 7f7d08fe8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:14.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.036+0000 7f7d097e9640 1 --2- 172.21.15.132:0/2735114572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d040b24b0 0x7f7d04131170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.036+0000 7f7d097e9640 10 monclient(hunting): get_auth_request con 0x7f7d040b24b0 auth_method 0 2024-03-13T08:11:14.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.036+0000 7f7d097e9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:14.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.036+0000 7f7d097e9640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:14.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.036+0000 7f7d097e9640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:14.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.036+0000 7f7d09fea640 1 --2- 172.21.15.132:0/2735114572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d04140120 0x7f7d0413e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.037+0000 7f7d09fea640 10 monclient(hunting): get_auth_request con 0x7f7d04140120 auth_method 0 2024-03-13T08:11:14.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.037+0000 7f7d09fea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:14.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.037+0000 7f7d09fea640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:14.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.037+0000 7f7d09fea640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:14.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.037+0000 7f7d097e9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:14.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.037+0000 7f7d097e9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:14.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.037+0000 7f7d097e9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:14.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.037+0000 7f7d09fea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:14.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.037+0000 7f7d09fea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:14.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.037+0000 7f7d09fea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:14.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.038+0000 7f7d08fe8640 10 monclient(hunting): handle_auth_done global_id 18986 payload 824 2024-03-13T08:11:14.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.038+0000 7f7d08fe8640 1 -- 172.21.15.132:0/2735114572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d04140120 msgr2=0x7f7d0413e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.038+0000 7f7d08fe8640 1 --2- 172.21.15.132:0/2735114572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d04140120 0x7f7d0413e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.038+0000 7f7d08fe8640 1 -- 172.21.15.132:0/2735114572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d040b24b0 msgr2=0x7f7d04131170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.038+0000 7f7d08fe8640 1 --2- 172.21.15.132:0/2735114572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d040b24b0 0x7f7d04131170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.038+0000 7f7d08fe8640 10 monclient: _finish_hunting 0 2024-03-13T08:11:14.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.038+0000 7f7d08fe8640 1 monclient: found mon.noname-c 2024-03-13T08:11:14.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.038+0000 7f7d08fe8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:14.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.038+0000 7f7d08fe8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T08:11:14.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.038+0000 7f7d08fe8640 1 -- 172.21.15.132:0/2735114572 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0413f470 con 0x7f7d040b52a0 2024-03-13T08:11:14.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.040+0000 7f7d08fe8640 1 --2- 172.21.15.132:0/2735114572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d040b52a0 0x7f7d04144db0 secure :-1 s=READY pgs=3538 cs=0 l=1 rev1=1 crypto rx=0x7f7cf402bee0 tx=0x7f7cf40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:14.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.040+0000 7f7cf9ffb640 10 client.?.objecter ms_handle_connect 0x7f7d040b52a0 2024-03-13T08:11:14.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.040+0000 7f7cf9ffb640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:14.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.042+0000 7f7cf9ffb640 1 -- 172.21.15.132:0/2735114572 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4035040 con 0x7f7d040b52a0 2024-03-13T08:11:14.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.042+0000 7f7cf9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:14.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.042+0000 7f7cf9ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T08:11:14.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.042+0000 7f7cf9ffb640 10 monclient: dump: 2024-03-13T08:11:14.066 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:14.066 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:14.066 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.067 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.067 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:14.067 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.067 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:14.067 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:14.067 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:14.067 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.042+0000 7f7cf9ffb640 10 monclient: _finish_auth 0 2024-03-13T08:11:14.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.042+0000 7f7cf9ffb640 10 monclient: _check_auth_tickets 2024-03-13T08:11:14.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.042+0000 7f7cf9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:14.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.042+0000 7f7d0ba4d640 5 monclient: authenticate success, global_id 18986 2024-03-13T08:11:14.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.042+0000 7f7d0ba4d640 10 monclient: _renew_subs 2024-03-13T08:11:14.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.042+0000 7f7d0ba4d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:14.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.042+0000 7f7d0ba4d640 1 -- 172.21.15.132:0/2735114572 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0413e250 con 0x7f7d040b52a0 2024-03-13T08:11:14.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.042+0000 7f7d0ba4d640 10 client.18986.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:14.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.042+0000 7f7d0ba4d640 10 monclient: _renew_subs 2024-03-13T08:11:14.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.042+0000 7f7d0ba4d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:14.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.042+0000 7f7d0ba4d640 1 -- 172.21.15.132:0/2735114572 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d04144680 con 0x7f7d040b52a0 2024-03-13T08:11:14.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.042+0000 7f7d0ba4d640 1 librados: init done 2024-03-13T08:11:14.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.043+0000 7f7d0ba4d640 10 client.18986.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:14.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.043+0000 7f7d0ba4d640 15 client.18986.objecter _session_command_op_assign -1 1 2024-03-13T08:11:14.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.043+0000 7f7d0ba4d640 20 client.18986.objecter _get_session osd=-1 returning homeless 2024-03-13T08:11:14.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.043+0000 7f7d0ba4d640 10 client.18986.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:14.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.043+0000 7f7d0ba4d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:14.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.043+0000 7f7d0ba4d640 1 -- 172.21.15.132:0/2735114572 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7d0404f000 con 0x7f7d040b52a0 2024-03-13T08:11:14.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.064+0000 7f7cf9ffb640 1 -- 172.21.15.132:0/2735114572 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0404f000 con 0x7f7d040b52a0 2024-03-13T08:11:14.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.064+0000 7f7cf9ffb640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:14.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.064+0000 7f7cf9ffb640 1 -- 172.21.15.132:0/2735114572 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf403bb00 con 0x7f7d040b52a0 2024-03-13T08:11:14.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.064+0000 7f7cf9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:14.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.064+0000 7f7cf9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T08:11:14.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.064+0000 7f7cf9ffb640 10 monclient: dump: 2024-03-13T08:11:14.069 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:14.069 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:14.069 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.069 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.069 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:14.069 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.069 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:14.069 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:14.070 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:14.070 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.064+0000 7f7cfb7fe640 4 set_mon_vals no callback set 2024-03-13T08:11:14.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.064+0000 7f7cf9ffb640 1 -- 172.21.15.132:0/2735114572 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f7cf4038020 con 0x7f7d040b52a0 2024-03-13T08:11:14.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.065+0000 7f7cf9ffb640 1 --2- 172.21.15.132:0/2735114572 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f7ce806d560 0x7f7ce806fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.065+0000 7f7d097e9640 1 --2- 172.21.15.132:0/2735114572 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f7ce806d560 0x7f7ce806fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.065+0000 7f7d097e9640 10 monclient: get_auth_request con 0x7f7ce806d560 auth_method 0 2024-03-13T08:11:14.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.065+0000 7f7cf9ffb640 10 client.18986.objecter ms_handle_connect 0x7f7ce806d560 2024-03-13T08:11:14.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.065+0000 7f7d097e9640 1 --2- 172.21.15.132:0/2735114572 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f7ce806d560 0x7f7ce806fa20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7d00000f60 tx=0x7f7d00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:14.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 1 -- 172.21.15.132:0/2735114572 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f7cf40790c0 con 0x7f7d040b52a0 2024-03-13T08:11:14.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 10 client.18986.objecter ms_dispatch 0x7f7d04000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:14.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 3 client.18986.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:14.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 3 client.18986.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:14.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 10 client.18986.objecter checking command 1 2024-03-13T08:11:14.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 1 --2- 172.21.15.132:0/2735114572 >> [v2:172.21.15.132:6826/958254560,v1:172.21.15.132:6827/958254560] conn(0x7f7ce8075230 0x7f7ce80776d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 20 client.18986.objecter _get_session s=0x7f7ce8074b60 osd=2 3 2024-03-13T08:11:14.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 20 client.18986.objecter put_session s=0x7f7ce8074b60 osd=2 3 2024-03-13T08:11:14.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 15 client.18986.objecter _session_command_op_remove -1 1 2024-03-13T08:11:14.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 20 client.18986.objecter _get_session s=0x7f7ce8074b60 osd=2 3 2024-03-13T08:11:14.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 20 client.18986.objecter get_session s=0x7f7ce8074b60 osd=2 3 2024-03-13T08:11:14.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 15 client.18986.objecter _session_command_op_assign 2 1 2024-03-13T08:11:14.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 20 client.18986.objecter put_session s=0x7f7ce8074b60 osd=2 4 2024-03-13T08:11:14.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 10 client.18986.objecter _send_command 1 2024-03-13T08:11:14.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 1 -- 172.21.15.132:0/2735114572 --> [v2:172.21.15.132:6826/958254560,v1:172.21.15.132:6827/958254560] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7ce8077da0 con 0x7f7ce8075230 2024-03-13T08:11:14.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 20 client.18986.objecter dump_active .. 0 homeless 2024-03-13T08:11:14.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 1 -- 172.21.15.132:0/2735114572 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_get_version_reply(handle=1 version=1293) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cf40772c0 con 0x7f7d040b52a0 2024-03-13T08:11:14.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7cf9ffb640 10 monclient: handle_get_version_reply finishing 1 version 1293 2024-03-13T08:11:14.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.069+0000 7f7d09fea640 1 --2- 172.21.15.132:0/2735114572 >> [v2:172.21.15.132:6826/958254560,v1:172.21.15.132:6827/958254560] conn(0x7f7ce8075230 0x7f7ce80776d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.070+0000 7f7d09fea640 10 monclient: get_auth_request con 0x7f7ce8075230 auth_method 0 2024-03-13T08:11:14.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.072+0000 7f7d09fea640 1 --2- 172.21.15.132:0/2735114572 >> [v2:172.21.15.132:6826/958254560,v1:172.21.15.132:6827/958254560] conn(0x7f7ce8075230 0x7f7ce80776d0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:14.074 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.072+0000 7f7cf9ffb640 10 client.18986.objecter ms_handle_connect 0x7f7ce8075230 2024-03-13T08:11:14.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.096+0000 7f7cf9ffb640 1 -- 172.21.15.132:0/2735114572 <== osd.2 v2:172.21.15.132:6826/958254560 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7ce8077da0 con 0x7f7ce8075230 2024-03-13T08:11:14.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.096+0000 7f7cf9ffb640 10 client.18986.objecter ms_dispatch 0x7f7d04000b90 command_reply(tid 1: 0 ) v1 2024-03-13T08:11:14.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.096+0000 7f7cf9ffb640 10 client.18986.objecter _finish_command 1 = system:0 2024-03-13T08:11:14.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.096+0000 7f7cf9ffb640 20 client.18986.objecter put_session s=0x7f7ce8074b60 osd=2 4 2024-03-13T08:11:14.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.096+0000 7f7cf9ffb640 15 client.18986.objecter _session_command_op_remove 2 1 2024-03-13T08:11:14.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.109+0000 7f7d0ba4d640 10 client.18986.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-03-13T08:11:14.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.109+0000 7f7d0ba4d640 15 client.18986.objecter _session_command_op_assign -1 2 2024-03-13T08:11:14.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.109+0000 7f7d0ba4d640 20 client.18986.objecter _get_session s=0x7f7ce8074b60 osd=2 3 2024-03-13T08:11:14.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.109+0000 7f7d0ba4d640 20 client.18986.objecter put_session s=0x7f7ce8074b60 osd=2 3 2024-03-13T08:11:14.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.109+0000 7f7d0ba4d640 20 client.18986.objecter _get_session s=0x7f7ce8074b60 osd=2 3 2024-03-13T08:11:14.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.109+0000 7f7d0ba4d640 15 client.18986.objecter _session_command_op_remove -1 2 2024-03-13T08:11:14.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.109+0000 7f7d0ba4d640 20 client.18986.objecter get_session s=0x7f7ce8074b60 osd=2 3 2024-03-13T08:11:14.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.109+0000 7f7d0ba4d640 15 client.18986.objecter _session_command_op_assign 2 2 2024-03-13T08:11:14.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.109+0000 7f7d0ba4d640 20 client.18986.objecter put_session s=0x7f7ce8074b60 osd=2 4 2024-03-13T08:11:14.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.109+0000 7f7d0ba4d640 10 client.18986.objecter _send_command 2 2024-03-13T08:11:14.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.109+0000 7f7d0ba4d640 1 -- 172.21.15.132:0/2735114572 --> [v2:172.21.15.132:6826/958254560,v1:172.21.15.132:6827/958254560] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7d0414dda0 con 0x7f7ce8075230 2024-03-13T08:11:14.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.124+0000 7f7cf9ffb640 1 -- 172.21.15.132:0/2735114572 <== osd.2 v2:172.21.15.132:6826/958254560 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7d0414dda0 con 0x7f7ce8075230 2024-03-13T08:11:14.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.124+0000 7f7cf9ffb640 10 client.18986.objecter ms_dispatch 0x7f7d04000b90 command_reply(tid 2: 0 ) v1 2024-03-13T08:11:14.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.124+0000 7f7cf9ffb640 10 client.18986.objecter _finish_command 2 = system:0 2024-03-13T08:11:14.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.124+0000 7f7cf9ffb640 20 client.18986.objecter put_session s=0x7f7ce8074b60 osd=2 4 2024-03-13T08:11:14.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.124+0000 7f7cf9ffb640 15 client.18986.objecter _session_command_op_remove 2 2 2024-03-13T08:11:14.125 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-13T08:11:14.126 INFO:teuthology.orchestra.run.smithi132.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-13T08:11:14.126 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-13T08:11:14.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 10 librados: watch_flush enter 2024-03-13T08:11:14.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 10 librados: watch_flush exit 2024-03-13T08:11:14.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 10 client.18986.objecter close_session for osd.2 2024-03-13T08:11:14.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 1 -- 172.21.15.132:0/2735114572 >> [v2:172.21.15.132:6826/958254560,v1:172.21.15.132:6827/958254560] conn(0x7f7ce8075230 msgr2=0x7f7ce80776d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 1 --2- 172.21.15.132:0/2735114572 >> [v2:172.21.15.132:6826/958254560,v1:172.21.15.132:6827/958254560] conn(0x7f7ce8075230 0x7f7ce80776d0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 20 client.18986.objecter put_session s=0x7f7ce8074b60 osd=2 1 2024-03-13T08:11:14.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 20 client.18986.objecter shutdown clearing up homeless session... 2024-03-13T08:11:14.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 10 client.18986.objecter successfully canceled tick 2024-03-13T08:11:14.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 1 -- 172.21.15.132:0/2735114572 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f7ce806d560 msgr2=0x7f7ce806fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.128 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 1 --2- 172.21.15.132:0/2735114572 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f7ce806d560 0x7f7ce806fa20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7d00000f60 tx=0x7f7d00009040 comp rx=0 tx=0).stop 2024-03-13T08:11:14.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 10 monclient: shutdown 2024-03-13T08:11:14.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:14.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 1 -- 172.21.15.132:0/2735114572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d040b52a0 msgr2=0x7f7d04144db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 1 --2- 172.21.15.132:0/2735114572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d040b52a0 0x7f7d04144db0 secure :-1 s=READY pgs=3538 cs=0 l=1 rev1=1 crypto rx=0x7f7cf402bee0 tx=0x7f7cf40046c0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.129 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d09fea640 1 -- 172.21.15.132:0/2735114572 reap_dead start 2024-03-13T08:11:14.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 1 -- 172.21.15.132:0/2735114572 shutdown_connections 2024-03-13T08:11:14.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 1 -- 172.21.15.132:0/2735114572 >> 172.21.15.132:0/2735114572 conn(0x7f7d04066e20 msgr2=0x7f7d040b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:14.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 1 -- 172.21.15.132:0/2735114572 shutdown_connections 2024-03-13T08:11:14.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 1 -- 172.21.15.132:0/2735114572 wait complete. 2024-03-13T08:11:14.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.125+0000 7f7d0ba4d640 1 librados: shutdown 2024-03-13T08:11:14.136 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-13T08:11:14.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.236+0000 7f31ed4ee640 10 monclient: get_monmap_and_config 2024-03-13T08:11:14.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.236+0000 7f31ed4ee640 10 monclient: build_initial_monmap 2024-03-13T08:11:14.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.236+0000 7f31ed4ee640 10 monclient: monmap: 2024-03-13T08:11:14.238 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:14.238 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:14.238 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:14.237706+0000 2024-03-13T08:11:14.238 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:14.237706+0000 2024-03-13T08:11:14.238 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:14.238 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.238 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:14.239 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:14.239 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:14.239 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:14.239 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:14.239 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.237+0000 7f31ed4ee640 1 Processor -- start 2024-03-13T08:11:14.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.237+0000 7f31ed4ee640 1 -- start start 2024-03-13T08:11:14.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.237+0000 7f31ed4ee640 10 monclient: init 2024-03-13T08:11:14.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.238+0000 7f31ed4ee640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:14.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.238+0000 7f31ed4ee640 10 monclient: _add_conns ranks=[1,4,0,3,2] 2024-03-13T08:11:14.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.238+0000 7f31ed4ee640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f31e80b1e70 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:14.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.238+0000 7f31ed4ee640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f31e80b00f0 0x7f31e813fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.238+0000 7f31ed4ee640 10 monclient(hunting): picked mon.noname-c con 0x7f31e80b00f0 addr v2:172.21.15.195:3300/0 2024-03-13T08:11:14.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.238+0000 7f31ed4ee640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f31e8140400 addr v1:172.21.15.132:6789/0 2024-03-13T08:11:14.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.238+0000 7f31ed4ee640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e80b1390 con 0x7f31e8140400 2024-03-13T08:11:14.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.238+0000 7f31ed4ee640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e80b1670 con 0x7f31e80b1e70 2024-03-13T08:11:14.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.238+0000 7f31ed4ee640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:14.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.238+0000 7f31ed4ee640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e80b05a0 con 0x7f31e80b00f0 2024-03-13T08:11:14.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.238+0000 7f31ed4ee640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:14.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.238+0000 7f31ed4ee640 10 monclient(hunting): authenticate will time out at 3501.646334s 2024-03-13T08:11:14.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.238+0000 7f31e77fe640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f31e8140400 0x7f31e8142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:60818/0 (socket says 172.21.15.132:60818) 2024-03-13T08:11:14.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.238+0000 7f31e77fe640 1 -- 172.21.15.132:0/3714889329 learned_addr learned my addr 172.21.15.132:0/3714889329 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T08:11:14.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.239+0000 7f31e67fc640 1 --2- 172.21.15.132:0/3714889329 >> v2:172.21.15.195:3300/0 conn(0x7f31e80b00f0 0x7f31e813fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.239+0000 7f31e67fc640 10 monclient(hunting): get_auth_request con 0x7f31e80b00f0 auth_method 0 2024-03-13T08:11:14.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.239+0000 7f31e67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:14.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.239+0000 7f31e67fc640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:14.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.239+0000 7f31e67fc640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:14.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.239+0000 7f31e67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:14.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.239+0000 7f31e67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:14.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.239+0000 7f31e67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:14.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.241+0000 7f31e5ffb640 1 -- 172.21.15.132:0/3714889329 <== mon.0 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443067412 0 0) 0x7f31e80b1390 con 0x7f31e8140400 2024-03-13T08:11:14.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.241+0000 7f31e5ffb640 10 monclient(hunting): _negotiate 2024-03-13T08:11:14.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.241+0000 7f31e5ffb640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:14.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.241+0000 7f31e5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:14.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.241+0000 7f31e5ffb640 10 monclient(hunting): my global_id is 16038 2024-03-13T08:11:14.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.241+0000 7f31e5ffb640 1 -- 172.21.15.132:0/3714889329 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d0003520 con 0x7f31e8140400 2024-03-13T08:11:14.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.241+0000 7f31e67fc640 10 monclient(hunting): handle_auth_done global_id 19681 payload 274 2024-03-13T08:11:14.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.241+0000 7f31e67fc640 1 -- 172.21.15.132:0/3714889329 >> v1:172.21.15.195:6789/0 conn(0x7f31e80b1e70 legacy=0x7f31e80b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:14.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.241+0000 7f31e67fc640 1 -- 172.21.15.132:0/3714889329 >> v1:172.21.15.132:6789/0 conn(0x7f31e8140400 legacy=0x7f31e8142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.241+0000 7f31e67fc640 10 monclient: _finish_hunting 0 2024-03-13T08:11:14.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.241+0000 7f31e67fc640 1 monclient: found mon.noname-c 2024-03-13T08:11:14.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.241+0000 7f31e67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:14.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.241+0000 7f31e67fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T08:11:14.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.241+0000 7f31e67fc640 1 -- 172.21.15.132:0/3714889329 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e81431e0 con 0x7f31e80b00f0 2024-03-13T08:11:14.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.244+0000 7f31e67fc640 1 --2- 172.21.15.132:0/3714889329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31e80b00f0 0x7f31e813fec0 secure :-1 s=READY pgs=4732 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0097a0 tx=0x7f31dc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec31975d26ce1c4b server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31e5ffb640 1 -- 172.21.15.132:0/3714889329 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc035020 con 0x7f31e80b00f0 2024-03-13T08:11:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31e5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T08:11:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31e5ffb640 10 monclient: dump: 2024-03-13T08:11:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:14.267 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:14.268 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31e5ffb640 10 monclient: _finish_auth 0 2024-03-13T08:11:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31e5ffb640 10 monclient: _check_auth_tickets 2024-03-13T08:11:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31e5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31e5ffb640 1 -- 172.21.15.132:0/3714889329 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc031830 con 0x7f31e80b00f0 2024-03-13T08:11:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31e5ffb640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31e5ffb640 1 -- 172.21.15.132:0/3714889329 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc031c00 con 0x7f31e80b00f0 2024-03-13T08:11:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31e5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31e5ffb640 10 monclient: dump: 2024-03-13T08:11:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:14.269 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31ed4ee640 5 monclient: authenticate success, global_id 19681 2024-03-13T08:11:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31ed4ee640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31ed4ee640 4 set_mon_vals no callback set 2024-03-13T08:11:14.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31ed4ee640 10 monclient: shutdown 2024-03-13T08:11:14.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31ed4ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:14.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31ed4ee640 1 -- 172.21.15.132:0/3714889329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31e80b00f0 msgr2=0x7f31e813fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31ed4ee640 1 --2- 172.21.15.132:0/3714889329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31e80b00f0 0x7f31e813fec0 secure :-1 s=READY pgs=4732 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0097a0 tx=0x7f31dc02b8d0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31ed4ee640 1 -- 172.21.15.132:0/3714889329 shutdown_connections 2024-03-13T08:11:14.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31ed4ee640 1 --2- 172.21.15.132:0/3714889329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31e80b00f0 0x7f31e813fec0 unknown :-1 s=CLOSED pgs=4732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31ed4ee640 1 -- 172.21.15.132:0/3714889329 >> 172.21.15.132:0/3714889329 conn(0x7f31e80b80e0 msgr2=0x7f31e80b84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:14.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31ed4ee640 1 -- 172.21.15.132:0/3714889329 shutdown_connections 2024-03-13T08:11:14.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.266+0000 7f31ed4ee640 1 -- 172.21.15.132:0/3714889329 wait complete. 2024-03-13T08:11:14.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.267+0000 7f31ed4ee640 10 monclient: build_initial_monmap 2024-03-13T08:11:14.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.267+0000 7f31ed4ee640 10 monclient: monmap: 2024-03-13T08:11:14.271 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:14.271 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:14.271 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:14.271 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:14.271 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:14.271 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.271 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:14.271 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:14.271 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:14.271 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.267+0000 7f31ed4ee640 1 librados: starting msgr at 2024-03-13T08:11:14.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.267+0000 7f31ed4ee640 1 librados: starting objecter 2024-03-13T08:11:14.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.267+0000 7f31ed4ee640 1 Processor -- start 2024-03-13T08:11:14.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.267+0000 7f31ed4ee640 1 -- start start 2024-03-13T08:11:14.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.267+0000 7f31ed4ee640 1 librados: setting wanted keys 2024-03-13T08:11:14.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.267+0000 7f31ed4ee640 1 librados: calling monclient init 2024-03-13T08:11:14.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.267+0000 7f31ed4ee640 10 monclient: init 2024-03-13T08:11:14.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.268+0000 7f31ed4ee640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:14.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.268+0000 7f31ed4ee640 10 monclient: _add_conns ranks=[2,1,0] 2024-03-13T08:11:14.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.268+0000 7f31ed4ee640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31e80b00f0 0x7f31e80ab630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.268+0000 7f31ed4ee640 10 monclient(hunting): picked mon.noname-c con 0x7f31e80b00f0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:14.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.268+0000 7f31ed4ee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31e80b1e70 0x7f31e81d3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.268+0000 7f31ed4ee640 10 monclient(hunting): picked mon.noname-b con 0x7f31e80b1e70 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:14.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.268+0000 7f31ed4ee640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31e8140400 0x7f31e80ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.268+0000 7f31ed4ee640 10 monclient(hunting): picked mon.noname-a con 0x7f31e8140400 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:14.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.268+0000 7f31ed4ee640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:14.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.268+0000 7f31ed4ee640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e814e2a0 con 0x7f31e8140400 2024-03-13T08:11:14.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.268+0000 7f31ed4ee640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:14.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.268+0000 7f31ed4ee640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e81437d0 con 0x7f31e80b1e70 2024-03-13T08:11:14.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.268+0000 7f31ed4ee640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:14.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.268+0000 7f31ed4ee640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e814e420 con 0x7f31e80b00f0 2024-03-13T08:11:14.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.268+0000 7f31ed4ee640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:14.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.268+0000 7f31ed4ee640 10 monclient(hunting): authenticate will time out at 3501.676609s 2024-03-13T08:11:14.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.269+0000 7f31e6ffd640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31e80b00f0 0x7f31e80ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.269+0000 7f31e77fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31e8140400 0x7f31e80ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.269+0000 7f31e6ffd640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31e80b00f0 0x7f31e80ab630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:54762/0 (socket says 172.21.15.132:54762) 2024-03-13T08:11:14.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.269+0000 7f31e6ffd640 1 -- 172.21.15.132:0/2343161990 learned_addr learned my addr 172.21.15.132:0/2343161990 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:14.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.269+0000 7f31e6ffd640 10 monclient(hunting): get_auth_request con 0x7f31e80b00f0 auth_method 0 2024-03-13T08:11:14.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.269+0000 7f31e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:14.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.269+0000 7f31e6ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:14.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.269+0000 7f31e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:14.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.270+0000 7f31e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:14.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.270+0000 7f31e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:14.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.270+0000 7f31e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:14.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.270+0000 7f31e67fc640 1 --2- 172.21.15.132:0/2343161990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31e80b1e70 0x7f31e81d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.271+0000 7f31e67fc640 10 monclient(hunting): get_auth_request con 0x7f31e80b1e70 auth_method 0 2024-03-13T08:11:14.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.271+0000 7f31e67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:14.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.271+0000 7f31e67fc640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:14.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.271+0000 7f31e67fc640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:14.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.271+0000 7f31e67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:14.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.271+0000 7f31e67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:14.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.271+0000 7f31e67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:14.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.273+0000 7f31e6ffd640 10 monclient(hunting): handle_auth_done global_id 18989 payload 824 2024-03-13T08:11:14.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.273+0000 7f31e6ffd640 1 -- 172.21.15.132:0/2343161990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31e80b1e70 msgr2=0x7f31e81d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.273+0000 7f31e6ffd640 1 --2- 172.21.15.132:0/2343161990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31e80b1e70 0x7f31e81d3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.273+0000 7f31e6ffd640 1 -- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31e8140400 msgr2=0x7f31e80ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.273+0000 7f31e6ffd640 1 --2- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31e8140400 0x7f31e80ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.273+0000 7f31e6ffd640 10 monclient: _finish_hunting 0 2024-03-13T08:11:14.278 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.273+0000 7f31e6ffd640 1 monclient: found mon.noname-c 2024-03-13T08:11:14.278 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.273+0000 7f31e6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:14.278 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.273+0000 7f31e6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T08:11:14.278 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.273+0000 7f31e6ffd640 1 -- 172.21.15.132:0/2343161990 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e8144380 con 0x7f31e80b00f0 2024-03-13T08:11:14.278 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.275+0000 7f31e6ffd640 1 --2- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31e80b00f0 0x7f31e80ab630 secure :-1 s=READY pgs=3539 cs=0 l=1 rev1=1 crypto rx=0x7f31e000c440 tx=0x7f31e000c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:14.278 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.275+0000 7f31cf7fe640 10 client.?.objecter ms_handle_connect 0x7f31e80b00f0 2024-03-13T08:11:14.278 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.275+0000 7f31cf7fe640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:14.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31cf7fe640 1 -- 172.21.15.132:0/2343161990 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0017070 con 0x7f31e80b00f0 2024-03-13T08:11:14.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:14.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31cf7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T08:11:14.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31cf7fe640 10 monclient: dump: 2024-03-13T08:11:14.280 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:14.280 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:14.280 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.281 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.281 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:14.281 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.281 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:14.281 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:14.281 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:14.281 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31cf7fe640 10 monclient: _finish_auth 0 2024-03-13T08:11:14.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31cf7fe640 10 monclient: _check_auth_tickets 2024-03-13T08:11:14.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:14.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31cf7fe640 1 -- 172.21.15.132:0/2343161990 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e00035f0 con 0x7f31e80b00f0 2024-03-13T08:11:14.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31cf7fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:14.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31cf7fe640 1 -- 172.21.15.132:0/2343161990 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e00039e0 con 0x7f31e80b00f0 2024-03-13T08:11:14.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:14.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31cf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T08:11:14.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31cf7fe640 10 monclient: dump: 2024-03-13T08:11:14.282 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:14.282 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:14.282 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.282 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.282 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:14.282 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.282 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:14.282 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:14.282 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:14.282 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31ed4ee640 5 monclient: authenticate success, global_id 18989 2024-03-13T08:11:14.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31ed4ee640 10 monclient: _renew_subs 2024-03-13T08:11:14.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31ed4ee640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:14.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31ed4ee640 1 -- 172.21.15.132:0/2343161990 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e81446c0 con 0x7f31e80b00f0 2024-03-13T08:11:14.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.279+0000 7f31e57fa640 4 set_mon_vals no callback set 2024-03-13T08:11:14.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.280+0000 7f31ed4ee640 10 client.18989.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:14.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.280+0000 7f31ed4ee640 10 monclient: _renew_subs 2024-03-13T08:11:14.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.280+0000 7f31ed4ee640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:14.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.280+0000 7f31ed4ee640 1 -- 172.21.15.132:0/2343161990 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e80ae360 con 0x7f31e80b00f0 2024-03-13T08:11:14.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.280+0000 7f31ed4ee640 1 librados: init done 2024-03-13T08:11:14.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.281+0000 7f31ed4ee640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:14.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.281+0000 7f31ed4ee640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:14.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.281+0000 7f31ed4ee640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:14.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.281+0000 7f31ed4ee640 1 -- 172.21.15.132:0/2343161990 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e804f000 con 0x7f31e80b00f0 2024-03-13T08:11:14.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.300+0000 7f31cf7fe640 1 -- 172.21.15.132:0/2343161990 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f31e001a070 con 0x7f31e80b00f0 2024-03-13T08:11:14.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.301+0000 7f31cf7fe640 1 --2- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f31d406d5b0 0x7f31d406fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.301+0000 7f31e67fc640 1 --2- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f31d406d5b0 0x7f31d406fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.301+0000 7f31e67fc640 10 monclient: get_auth_request con 0x7f31d406d5b0 auth_method 0 2024-03-13T08:11:14.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.301+0000 7f31e67fc640 1 -- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f31d406d5b0 msgr2=0x7f31d406fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-13T08:11:14.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.301+0000 7f31e67fc640 1 -- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f31d406d5b0 msgr2=0x7f31d406fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-13T08:11:14.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.301+0000 7f31e67fc640 1 --2- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f31d406d5b0 0x7f31d406fa70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f31dc036000 tx=0x7f31dc03c040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-13T08:11:14.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.301+0000 7f31e67fc640 1 --2- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f31d406d5b0 0x7f31d406fa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T08:11:14.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.304+0000 7f31cf7fe640 1 -- 172.21.15.132:0/2343161990 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f31e00565e0 con 0x7f31e80b00f0 2024-03-13T08:11:14.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.304+0000 7f31cf7fe640 10 client.18989.objecter ms_dispatch 0x7f31e8000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:14.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.304+0000 7f31cf7fe640 3 client.18989.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:14.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.304+0000 7f31cf7fe640 3 client.18989.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:14.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.304+0000 7f31cf7fe640 20 client.18989.objecter dump_active .. 0 homeless 2024-03-13T08:11:14.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.502+0000 7f31e67fc640 1 --2- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f31d406d5b0 0x7f31d406fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.502+0000 7f31e67fc640 10 monclient: get_auth_request con 0x7f31d406d5b0 auth_method 0 2024-03-13T08:11:14.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.502+0000 7f31e67fc640 1 --2- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f31d406d5b0 0x7f31d406fa70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f31dc02b910 tx=0x7f31dc03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:14.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.502+0000 7f31cf7fe640 10 client.18989.objecter ms_handle_connect 0x7f31d406d5b0 2024-03-13T08:11:14.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.544+0000 7f31cf7fe640 1 -- 172.21.15.132:0/2343161990 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f31e0021080 con 0x7f31e80b00f0 2024-03-13T08:11:14.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.544+0000 7f31cf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:14.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.544+0000 7f31cf7fe640 10 monclient: _finish_command 1 = system:0 2024-03-13T08:11:14.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.728+0000 7f31ed4ee640 1 -- 172.21.15.132:0/2343161990 --> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f31e80b77c0 con 0x7f31d406d5b0 2024-03-13T08:11:14.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.728+0000 7f31cf7fe640 1 -- 172.21.15.132:0/2343161990 <== mgr.4104 v2:172.21.15.132:6800/3847550149 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f31e80b77c0 con 0x7f31d406d5b0 2024-03-13T08:11:14.729 INFO:teuthology.orchestra.run.smithi132.stderr:instructed osd(s) 2 to deep-scrub 2024-03-13T08:11:14.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 10 librados: watch_flush enter 2024-03-13T08:11:14.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 10 librados: watch_flush exit 2024-03-13T08:11:14.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 20 client.18989.objecter shutdown clearing up homeless session... 2024-03-13T08:11:14.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 10 client.18989.objecter successfully canceled tick 2024-03-13T08:11:14.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 1 -- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f31d406d5b0 msgr2=0x7f31d406fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 1 --2- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f31d406d5b0 0x7f31d406fa70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f31dc02b910 tx=0x7f31dc03c040 comp rx=0 tx=0).stop 2024-03-13T08:11:14.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 10 monclient: shutdown 2024-03-13T08:11:14.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:14.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 1 -- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31e80b00f0 msgr2=0x7f31e80ab630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 1 --2- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31e80b00f0 0x7f31e80ab630 secure :-1 s=READY pgs=3539 cs=0 l=1 rev1=1 crypto rx=0x7f31e000c440 tx=0x7f31e000c900 comp rx=0 tx=0).stop 2024-03-13T08:11:14.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 1 -- 172.21.15.132:0/2343161990 shutdown_connections 2024-03-13T08:11:14.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 1 --2- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f31d406d5b0 0x7f31d406fa70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 1 --2- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31e8140400 0x7f31e80ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 1 --2- 172.21.15.132:0/2343161990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31e80b1e70 0x7f31e81d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 1 --2- 172.21.15.132:0/2343161990 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31e80b00f0 0x7f31e80ab630 unknown :-1 s=CLOSED pgs=3539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 1 -- 172.21.15.132:0/2343161990 >> 172.21.15.132:0/2343161990 conn(0x7f31e80b80e0 msgr2=0x7f31e80b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:14.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.730+0000 7f31ed4ee640 1 -- 172.21.15.132:0/2343161990 shutdown_connections 2024-03-13T08:11:14.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.731+0000 7f31ed4ee640 1 -- 172.21.15.132:0/2343161990 wait complete. 2024-03-13T08:11:14.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.731+0000 7f31ed4ee640 1 librados: shutdown 2024-03-13T08:11:14.743 INFO:tasks.ceph:Scrubbing osd.3 2024-03-13T08:11:14.743 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-13T08:11:14.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.849+0000 7f4cdcd6a640 10 monclient: get_monmap_and_config 2024-03-13T08:11:14.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.849+0000 7f4cdcd6a640 10 monclient: build_initial_monmap 2024-03-13T08:11:14.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.849+0000 7f4cdcd6a640 10 monclient: monmap: 2024-03-13T08:11:14.850 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:14.850 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:14.850 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:14.850355+0000 2024-03-13T08:11:14.850 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:14.850355+0000 2024-03-13T08:11:14.850 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:14.850 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.850 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:14.851 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:14.851 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:14.851 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:14.851 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:14.851 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.850+0000 7f4cdcd6a640 1 Processor -- start 2024-03-13T08:11:14.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.850+0000 7f4cdcd6a640 1 -- start start 2024-03-13T08:11:14.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.850+0000 7f4cdcd6a640 10 monclient: init 2024-03-13T08:11:14.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cdcd6a640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:14.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cdcd6a640 10 monclient: _add_conns ranks=[3,1,4,2,0] 2024-03-13T08:11:14.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cdcd6a640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4cd80b52a0 0x7f4cd813fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cdcd6a640 10 monclient(hunting): picked mon.noname-a con 0x7f4cd80b52a0 addr v2:172.21.15.132:3300/0 2024-03-13T08:11:14.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cdcd6a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4cd80b24b0 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:14.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cdcd6a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4cd80a9ef0 0x7f4cd813ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cdcd6a640 10 monclient(hunting): picked mon.noname-c con 0x7f4cd80a9ef0 addr v2:172.21.15.195:3300/0 2024-03-13T08:11:14.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cdcd6a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd80b4790 con 0x7f4cd80b24b0 2024-03-13T08:11:14.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cdcd6a640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:14.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cdcd6a640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd80b5680 con 0x7f4cd80b52a0 2024-03-13T08:11:14.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cdcd6a640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:14.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cdcd6a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd80aa4b0 con 0x7f4cd80a9ef0 2024-03-13T08:11:14.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cdcd6a640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:14.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cdcd6a640 10 monclient(hunting): authenticate will time out at 3502.259196s 2024-03-13T08:11:14.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cd659c640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4cd80b52a0 0x7f4cd813fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cd659c640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4cd80b52a0 0x7f4cd813fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51178/0 (socket says 172.21.15.132:51178) 2024-03-13T08:11:14.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cd659c640 1 -- 172.21.15.132:0/1893841095 learned_addr learned my addr 172.21.15.132:0/1893841095 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:14.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cd659c640 10 monclient(hunting): get_auth_request con 0x7f4cd80b52a0 auth_method 0 2024-03-13T08:11:14.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cd659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:14.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cd659c640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:14.854 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cd659c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:14.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.851+0000 7f4cd6d9d640 1 --2- 172.21.15.132:0/1893841095 >> v2:172.21.15.195:3300/0 conn(0x7f4cd80a9ef0 0x7f4cd813ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.852+0000 7f4cd6d9d640 10 monclient(hunting): get_auth_request con 0x7f4cd80a9ef0 auth_method 0 2024-03-13T08:11:14.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.852+0000 7f4cd6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:14.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.852+0000 7f4cd6d9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:14.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.852+0000 7f4cd6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:14.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.852+0000 7f4cd659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:14.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.852+0000 7f4cd659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:14.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.852+0000 7f4cd659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:14.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.852+0000 7f4cd6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:14.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.852+0000 7f4cd6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:14.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.852+0000 7f4cd6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:14.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.853+0000 7f4cd659c640 10 monclient(hunting): handle_auth_done global_id 16041 payload 274 2024-03-13T08:11:14.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.854+0000 7f4cd659c640 1 -- 172.21.15.132:0/1893841095 >> v2:172.21.15.195:3300/0 conn(0x7f4cd80a9ef0 msgr2=0x7f4cd813ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:14.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.854+0000 7f4cd659c640 1 --2- 172.21.15.132:0/1893841095 >> v2:172.21.15.195:3300/0 conn(0x7f4cd80a9ef0 0x7f4cd813ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.854+0000 7f4cd659c640 1 -- 172.21.15.132:0/1893841095 >> v1:172.21.15.195:6789/0 conn(0x7f4cd80b24b0 legacy=0x7f4cd8145c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:14.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.854+0000 7f4cd659c640 10 monclient: _finish_hunting 0 2024-03-13T08:11:14.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.854+0000 7f4cd659c640 1 monclient: found mon.noname-a 2024-03-13T08:11:14.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.854+0000 7f4cd659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:14.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.854+0000 7f4cd659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T08:11:14.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.854+0000 7f4cd659c640 1 -- 172.21.15.132:0/1893841095 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd8146390 con 0x7f4cd80b52a0 2024-03-13T08:11:14.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.855+0000 7f4cd659c640 1 --2- 172.21.15.132:0/1893841095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd80b52a0 0x7f4cd813fa10 secure :-1 s=READY pgs=3265 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80aa420 tx=0x7f4cc802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375a354829583c89 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:14.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cd5d9b640 1 -- 172.21.15.132:0/1893841095 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc802e070 con 0x7f4cd80b52a0 2024-03-13T08:11:14.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cd5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:14.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cd5d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T08:11:14.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cd5d9b640 10 monclient: dump: 2024-03-13T08:11:14.859 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:14.860 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:14.860 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.860 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.860 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:14.860 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.860 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:14.860 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:14.860 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:14.860 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cd5d9b640 10 monclient: _finish_auth 0 2024-03-13T08:11:14.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cd5d9b640 10 monclient: _check_auth_tickets 2024-03-13T08:11:14.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cd5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:14.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cd5d9b640 1 -- 172.21.15.132:0/1893841095 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc8037070 con 0x7f4cd80b52a0 2024-03-13T08:11:14.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cdcd6a640 5 monclient: authenticate success, global_id 16041 2024-03-13T08:11:14.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cdcd6a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T08:11:14.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cd5d9b640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:14.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cd5d9b640 1 -- 172.21.15.132:0/1893841095 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8031a00 con 0x7f4cd80b52a0 2024-03-13T08:11:14.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cdcd6a640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:14.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cdcd6a640 4 set_mon_vals no callback set 2024-03-13T08:11:14.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cd5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:14.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cd5d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T08:11:14.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.859+0000 7f4cd5d9b640 10 monclient: dump: 2024-03-13T08:11:14.861 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:14.861 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:14.861 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.861 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.861 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:14.861 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.862 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:14.862 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:14.862 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:14.862 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.860+0000 7f4cdcd6a640 10 monclient: shutdown 2024-03-13T08:11:14.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.860+0000 7f4cdcd6a640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:14.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.860+0000 7f4cdcd6a640 1 -- 172.21.15.132:0/1893841095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd80b52a0 msgr2=0x7f4cd813fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.860+0000 7f4cdcd6a640 1 --2- 172.21.15.132:0/1893841095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd80b52a0 0x7f4cd813fa10 secure :-1 s=READY pgs=3265 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80aa420 tx=0x7f4cc802b880 comp rx=0 tx=0).stop 2024-03-13T08:11:14.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.860+0000 7f4cdcd6a640 1 -- 172.21.15.132:0/1893841095 shutdown_connections 2024-03-13T08:11:14.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.860+0000 7f4cdcd6a640 1 --2- 172.21.15.132:0/1893841095 >> v2:172.21.15.195:3300/0 conn(0x7f4cd80a9ef0 0x7f4cd813ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.860+0000 7f4cdcd6a640 1 --2- 172.21.15.132:0/1893841095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd80b52a0 0x7f4cd813fa10 unknown :-1 s=CLOSED pgs=3265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.860+0000 7f4cdcd6a640 1 -- 172.21.15.132:0/1893841095 >> 172.21.15.132:0/1893841095 conn(0x7f4cd8066e20 msgr2=0x7f4cd8067230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:14.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.860+0000 7f4cdcd6a640 1 -- 172.21.15.132:0/1893841095 shutdown_connections 2024-03-13T08:11:14.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.860+0000 7f4cdcd6a640 1 -- 172.21.15.132:0/1893841095 wait complete. 2024-03-13T08:11:14.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.861+0000 7f4cdcd6a640 10 monclient: build_initial_monmap 2024-03-13T08:11:14.867 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.861+0000 7f4cdcd6a640 10 monclient: monmap: 2024-03-13T08:11:14.867 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:14.867 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:14.867 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:14.867 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:14.867 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:14.867 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.867 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:14.867 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:14.867 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:14.867 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.867 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.861+0000 7f4cdcd6a640 1 librados: starting msgr at 2024-03-13T08:11:14.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.861+0000 7f4cdcd6a640 1 librados: starting objecter 2024-03-13T08:11:14.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.861+0000 7f4cdcd6a640 1 Processor -- start 2024-03-13T08:11:14.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.861+0000 7f4cdcd6a640 1 -- start start 2024-03-13T08:11:14.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.861+0000 7f4cdcd6a640 1 librados: setting wanted keys 2024-03-13T08:11:14.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.861+0000 7f4cdcd6a640 1 librados: calling monclient init 2024-03-13T08:11:14.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.861+0000 7f4cdcd6a640 10 monclient: init 2024-03-13T08:11:14.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.861+0000 7f4cdcd6a640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:14.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.861+0000 7f4cdcd6a640 10 monclient: _add_conns ranks=[1,2,0] 2024-03-13T08:11:14.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.861+0000 7f4cdcd6a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cd80a9ef0 0x7f4cd8140580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.861+0000 7f4cdcd6a640 10 monclient(hunting): picked mon.noname-b con 0x7f4cd80a9ef0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:14.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cdcd6a640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cd80b24b0 0x7f4cd8144d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cdcd6a640 10 monclient(hunting): picked mon.noname-c con 0x7f4cd80b24b0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:14.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cdcd6a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd80b52a0 0x7f4cd81320c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cdcd6a640 10 monclient(hunting): picked mon.noname-a con 0x7f4cd80b52a0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:14.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cdcd6a640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:14.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cdcd6a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd814d130 con 0x7f4cd80b52a0 2024-03-13T08:11:14.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cdcd6a640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:14.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cdcd6a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd814cfb0 con 0x7f4cd80a9ef0 2024-03-13T08:11:14.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cdcd6a640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:14.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cdcd6a640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd814d2b0 con 0x7f4cd80b24b0 2024-03-13T08:11:14.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cdcd6a640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:14.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cdcd6a640 10 monclient(hunting): authenticate will time out at 3502.270140s 2024-03-13T08:11:14.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cd6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd80b52a0 0x7f4cd81320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cd659c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cd80a9ef0 0x7f4cd8140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4ccffff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cd80b24b0 0x7f4cd8144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cd6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd80b52a0 0x7f4cd81320c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51186/0 (socket says 172.21.15.132:51186) 2024-03-13T08:11:14.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cd6d9d640 1 -- 172.21.15.132:0/2614811119 learned_addr learned my addr 172.21.15.132:0/2614811119 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:14.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cd6d9d640 10 monclient(hunting): get_auth_request con 0x7f4cd80b52a0 auth_method 0 2024-03-13T08:11:14.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cd6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:14.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cd6d9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:14.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cd6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:14.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cd659c640 10 monclient(hunting): get_auth_request con 0x7f4cd80a9ef0 auth_method 0 2024-03-13T08:11:14.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cd659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:14.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cd659c640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:14.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.862+0000 7f4cd659c640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:14.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.863+0000 7f4ccffff640 10 monclient(hunting): get_auth_request con 0x7f4cd80b24b0 auth_method 0 2024-03-13T08:11:14.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.863+0000 7f4ccffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:14.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.863+0000 7f4ccffff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:14.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.863+0000 7f4ccffff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:14.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.863+0000 7f4cd6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:14.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.863+0000 7f4cd6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:14.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.863+0000 7f4cd6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:14.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.863+0000 7f4cd659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:14.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.863+0000 7f4cd659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:14.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.863+0000 7f4cd659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:14.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.863+0000 7f4ccffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:14.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.863+0000 7f4ccffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:14.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.863+0000 7f4ccffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:14.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.865+0000 7f4cd6d9d640 10 monclient(hunting): handle_auth_done global_id 16044 payload 824 2024-03-13T08:11:14.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.866+0000 7f4cd6d9d640 1 -- 172.21.15.132:0/2614811119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cd80b24b0 msgr2=0x7f4cd8144d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.866+0000 7f4cd6d9d640 1 --2- 172.21.15.132:0/2614811119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cd80b24b0 0x7f4cd8144d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.866+0000 7f4cd6d9d640 1 -- 172.21.15.132:0/2614811119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cd80a9ef0 msgr2=0x7f4cd8140580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.866+0000 7f4cd6d9d640 1 --2- 172.21.15.132:0/2614811119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cd80a9ef0 0x7f4cd8140580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.866+0000 7f4cd6d9d640 10 monclient: _finish_hunting 0 2024-03-13T08:11:14.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.866+0000 7f4cd6d9d640 1 monclient: found mon.noname-a 2024-03-13T08:11:14.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.866+0000 7f4cd6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:14.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.866+0000 7f4cd6d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T08:11:14.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.866+0000 7f4cd6d9d640 1 -- 172.21.15.132:0/2614811119 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd8134f30 con 0x7f4cd80b52a0 2024-03-13T08:11:14.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.866+0000 7f4cd659c640 1 --2- 172.21.15.132:0/2614811119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4cd80a9ef0 0x7f4cd8140580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-13T08:11:14.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.868+0000 7f4cd6d9d640 1 --2- 172.21.15.132:0/2614811119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd80b52a0 0x7f4cd81320c0 secure :-1 s=READY pgs=3266 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009dd0 tx=0x7f4cd000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:14.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.868+0000 7f4cceffd640 10 client.?.objecter ms_handle_connect 0x7f4cd80b52a0 2024-03-13T08:11:14.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.868+0000 7f4cceffd640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:14.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cceffd640 1 -- 172.21.15.132:0/2614811119 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0018020 con 0x7f4cd80b52a0 2024-03-13T08:11:14.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:14.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cceffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T08:11:14.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cceffd640 10 monclient: dump: 2024-03-13T08:11:14.876 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:14.876 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:14.876 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.876 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.876 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:14.876 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.876 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:14.876 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:14.876 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:14.876 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cceffd640 10 monclient: _finish_auth 0 2024-03-13T08:11:14.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cceffd640 10 monclient: _check_auth_tickets 2024-03-13T08:11:14.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:14.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cceffd640 1 -- 172.21.15.132:0/2614811119 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd0014480 con 0x7f4cd80b52a0 2024-03-13T08:11:14.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cceffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:14.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cceffd640 1 -- 172.21.15.132:0/2614811119 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0014870 con 0x7f4cd80b52a0 2024-03-13T08:11:14.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:14.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cceffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T08:11:14.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cceffd640 10 monclient: dump: 2024-03-13T08:11:14.877 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:14.877 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:14.877 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.877 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:14.877 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:14.877 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:14.878 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:14.878 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:14.878 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:14.878 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:14.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cdcd6a640 5 monclient: authenticate success, global_id 16044 2024-03-13T08:11:14.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cd559a640 4 set_mon_vals no callback set 2024-03-13T08:11:14.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cdcd6a640 10 monclient: _renew_subs 2024-03-13T08:11:14.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cdcd6a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T08:11:14.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cdcd6a640 1 -- 172.21.15.132:0/2614811119 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd8135210 con 0x7f4cd80b52a0 2024-03-13T08:11:14.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cdcd6a640 10 client.16044.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:14.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cdcd6a640 10 monclient: _renew_subs 2024-03-13T08:11:14.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cdcd6a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T08:11:14.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cdcd6a640 1 -- 172.21.15.132:0/2614811119 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd8132600 con 0x7f4cd80b52a0 2024-03-13T08:11:14.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.872+0000 7f4cdcd6a640 1 librados: init done 2024-03-13T08:11:14.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.873+0000 7f4cdcd6a640 10 client.16044.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:14.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.873+0000 7f4cdcd6a640 15 client.16044.objecter _session_command_op_assign -1 1 2024-03-13T08:11:14.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.873+0000 7f4cdcd6a640 20 client.16044.objecter _get_session osd=-1 returning homeless 2024-03-13T08:11:14.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.873+0000 7f4cdcd6a640 10 client.16044.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:14.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.873+0000 7f4cdcd6a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T08:11:14.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.873+0000 7f4cdcd6a640 1 -- 172.21.15.132:0/2614811119 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4cd804f000 con 0x7f4cd80b52a0 2024-03-13T08:11:14.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.892+0000 7f4cceffd640 1 -- 172.21.15.132:0/2614811119 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f4cd0005d10 con 0x7f4cd80b52a0 2024-03-13T08:11:14.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.892+0000 7f4cceffd640 1 --2- 172.21.15.132:0/2614811119 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4cb806d5b0 0x7f4cb806fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.893+0000 7f4cd659c640 1 --2- 172.21.15.132:0/2614811119 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4cb806d5b0 0x7f4cb806fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.893+0000 7f4cd659c640 10 monclient: get_auth_request con 0x7f4cb806d5b0 auth_method 0 2024-03-13T08:11:14.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.893+0000 7f4cd659c640 1 --2- 172.21.15.132:0/2614811119 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4cb806d5b0 0x7f4cb806fa70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8036000 tx=0x7f4cc803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:14.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.893+0000 7f4cceffd640 10 client.16044.objecter ms_handle_connect 0x7f4cb806d5b0 2024-03-13T08:11:14.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4cceffd640 1 -- 172.21.15.132:0/2614811119 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f4cd0058010 con 0x7f4cd80b52a0 2024-03-13T08:11:14.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4cceffd640 10 client.16044.objecter ms_dispatch 0x7f4cd8000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:14.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4cceffd640 3 client.16044.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:14.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4cceffd640 3 client.16044.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:14.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4cceffd640 10 client.16044.objecter checking command 1 2024-03-13T08:11:14.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4cceffd640 1 --2- 172.21.15.132:0/2614811119 >> [v2:172.21.15.132:6810/1770057764,v1:172.21.15.132:6811/1770057764] conn(0x7f4cb8075220 0x7f4cb80776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:14.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4cceffd640 20 client.16044.objecter _get_session s=0x7f4cb8074b50 osd=3 3 2024-03-13T08:11:14.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4cceffd640 20 client.16044.objecter put_session s=0x7f4cb8074b50 osd=3 3 2024-03-13T08:11:14.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4cceffd640 15 client.16044.objecter _session_command_op_remove -1 1 2024-03-13T08:11:14.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4cceffd640 20 client.16044.objecter _get_session s=0x7f4cb8074b50 osd=3 3 2024-03-13T08:11:14.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4cceffd640 20 client.16044.objecter get_session s=0x7f4cb8074b50 osd=3 3 2024-03-13T08:11:14.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4cceffd640 15 client.16044.objecter _session_command_op_assign 3 1 2024-03-13T08:11:14.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4cceffd640 20 client.16044.objecter put_session s=0x7f4cb8074b50 osd=3 4 2024-03-13T08:11:14.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4cceffd640 10 client.16044.objecter _send_command 1 2024-03-13T08:11:14.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4cceffd640 1 -- 172.21.15.132:0/2614811119 --> [v2:172.21.15.132:6810/1770057764,v1:172.21.15.132:6811/1770057764] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4cd00091c0 con 0x7f4cb8075220 2024-03-13T08:11:14.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4cceffd640 20 client.16044.objecter dump_active .. 0 homeless 2024-03-13T08:11:14.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.896+0000 7f4ccffff640 1 --2- 172.21.15.132:0/2614811119 >> [v2:172.21.15.132:6810/1770057764,v1:172.21.15.132:6811/1770057764] conn(0x7f4cb8075220 0x7f4cb80776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:14.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.897+0000 7f4ccffff640 10 monclient: get_auth_request con 0x7f4cb8075220 auth_method 0 2024-03-13T08:11:14.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.897+0000 7f4cceffd640 1 -- 172.21.15.132:0/2614811119 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=1293) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cd0056210 con 0x7f4cd80b52a0 2024-03-13T08:11:14.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.897+0000 7f4cceffd640 10 monclient: handle_get_version_reply finishing 1 version 1293 2024-03-13T08:11:14.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.900+0000 7f4ccffff640 1 --2- 172.21.15.132:0/2614811119 >> [v2:172.21.15.132:6810/1770057764,v1:172.21.15.132:6811/1770057764] conn(0x7f4cb8075220 0x7f4cb80776c0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:14.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.900+0000 7f4cceffd640 10 client.16044.objecter ms_handle_connect 0x7f4cb8075220 2024-03-13T08:11:14.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.923+0000 7f4cceffd640 1 -- 172.21.15.132:0/2614811119 <== osd.3 v2:172.21.15.132:6810/1770057764 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4cd00091c0 con 0x7f4cb8075220 2024-03-13T08:11:14.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.923+0000 7f4cceffd640 10 client.16044.objecter ms_dispatch 0x7f4cd8000b90 command_reply(tid 1: 0 ) v1 2024-03-13T08:11:14.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.923+0000 7f4cceffd640 10 client.16044.objecter _finish_command 1 = system:0 2024-03-13T08:11:14.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.923+0000 7f4cceffd640 20 client.16044.objecter put_session s=0x7f4cb8074b50 osd=3 4 2024-03-13T08:11:14.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.923+0000 7f4cceffd640 15 client.16044.objecter _session_command_op_remove 3 1 2024-03-13T08:11:14.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.937+0000 7f4cdcd6a640 10 client.16044.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-03-13T08:11:14.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.937+0000 7f4cdcd6a640 15 client.16044.objecter _session_command_op_assign -1 2 2024-03-13T08:11:14.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.937+0000 7f4cdcd6a640 20 client.16044.objecter _get_session s=0x7f4cb8074b50 osd=3 3 2024-03-13T08:11:14.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.937+0000 7f4cdcd6a640 20 client.16044.objecter put_session s=0x7f4cb8074b50 osd=3 3 2024-03-13T08:11:14.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.937+0000 7f4cdcd6a640 20 client.16044.objecter _get_session s=0x7f4cb8074b50 osd=3 3 2024-03-13T08:11:14.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.937+0000 7f4cdcd6a640 15 client.16044.objecter _session_command_op_remove -1 2 2024-03-13T08:11:14.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.937+0000 7f4cdcd6a640 20 client.16044.objecter get_session s=0x7f4cb8074b50 osd=3 3 2024-03-13T08:11:14.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.937+0000 7f4cdcd6a640 15 client.16044.objecter _session_command_op_assign 3 2 2024-03-13T08:11:14.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.937+0000 7f4cdcd6a640 20 client.16044.objecter put_session s=0x7f4cb8074b50 osd=3 4 2024-03-13T08:11:14.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.937+0000 7f4cdcd6a640 10 client.16044.objecter _send_command 2 2024-03-13T08:11:14.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.937+0000 7f4cdcd6a640 1 -- 172.21.15.132:0/2614811119 --> [v2:172.21.15.132:6810/1770057764,v1:172.21.15.132:6811/1770057764] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4cd81353e0 con 0x7f4cb8075220 2024-03-13T08:11:14.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.950+0000 7f4cceffd640 1 -- 172.21.15.132:0/2614811119 <== osd.3 v2:172.21.15.132:6810/1770057764 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4cd81353e0 con 0x7f4cb8075220 2024-03-13T08:11:14.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.950+0000 7f4cceffd640 10 client.16044.objecter ms_dispatch 0x7f4cd8000b90 command_reply(tid 2: 0 ) v1 2024-03-13T08:11:14.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.950+0000 7f4cceffd640 10 client.16044.objecter _finish_command 2 = system:0 2024-03-13T08:11:14.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.950+0000 7f4cceffd640 20 client.16044.objecter put_session s=0x7f4cb8074b50 osd=3 4 2024-03-13T08:11:14.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.950+0000 7f4cceffd640 15 client.16044.objecter _session_command_op_remove 3 2 2024-03-13T08:11:14.951 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-13T08:11:14.951 INFO:teuthology.orchestra.run.smithi132.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-13T08:11:14.951 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-13T08:11:14.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 10 librados: watch_flush enter 2024-03-13T08:11:14.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 10 librados: watch_flush exit 2024-03-13T08:11:14.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 10 client.16044.objecter close_session for osd.3 2024-03-13T08:11:14.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 1 -- 172.21.15.132:0/2614811119 >> [v2:172.21.15.132:6810/1770057764,v1:172.21.15.132:6811/1770057764] conn(0x7f4cb8075220 msgr2=0x7f4cb80776c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 1 --2- 172.21.15.132:0/2614811119 >> [v2:172.21.15.132:6810/1770057764,v1:172.21.15.132:6811/1770057764] conn(0x7f4cb8075220 0x7f4cb80776c0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:14.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 20 client.16044.objecter put_session s=0x7f4cb8074b50 osd=3 1 2024-03-13T08:11:14.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 20 client.16044.objecter shutdown clearing up homeless session... 2024-03-13T08:11:14.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 10 client.16044.objecter successfully canceled tick 2024-03-13T08:11:14.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 1 -- 172.21.15.132:0/2614811119 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4cb806d5b0 msgr2=0x7f4cb806fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 1 --2- 172.21.15.132:0/2614811119 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f4cb806d5b0 0x7f4cb806fa70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8036000 tx=0x7f4cc803d040 comp rx=0 tx=0).stop 2024-03-13T08:11:14.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 10 monclient: shutdown 2024-03-13T08:11:14.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:14.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 1 -- 172.21.15.132:0/2614811119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd80b52a0 msgr2=0x7f4cd81320c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:14.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 1 --2- 172.21.15.132:0/2614811119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd80b52a0 0x7f4cd81320c0 secure :-1 s=READY pgs=3266 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009dd0 tx=0x7f4cd000c490 comp rx=0 tx=0).stop 2024-03-13T08:11:14.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cd6d9d640 1 -- 172.21.15.132:0/2614811119 reap_dead start 2024-03-13T08:11:14.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 1 -- 172.21.15.132:0/2614811119 shutdown_connections 2024-03-13T08:11:14.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 1 -- 172.21.15.132:0/2614811119 >> 172.21.15.132:0/2614811119 conn(0x7f4cd8066e20 msgr2=0x7f4cd8139720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:14.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 1 -- 172.21.15.132:0/2614811119 shutdown_connections 2024-03-13T08:11:14.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 1 -- 172.21.15.132:0/2614811119 wait complete. 2024-03-13T08:11:14.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:14.951+0000 7f4cdcd6a640 1 librados: shutdown 2024-03-13T08:11:14.964 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-13T08:11:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.077+0000 7f007a8fd640 10 monclient: get_monmap_and_config 2024-03-13T08:11:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.077+0000 7f007a8fd640 10 monclient: build_initial_monmap 2024-03-13T08:11:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.077+0000 7f007a8fd640 10 monclient: monmap: 2024-03-13T08:11:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:15.078552+0000 2024-03-13T08:11:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:15.078552+0000 2024-03-13T08:11:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:15.079 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:15.079 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:15.079 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:15.079 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.078+0000 7f007a8fd640 1 Processor -- start 2024-03-13T08:11:15.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.078+0000 7f007a8fd640 1 -- start start 2024-03-13T08:11:15.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.078+0000 7f007a8fd640 10 monclient: init 2024-03-13T08:11:15.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.079+0000 7f007a8fd640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:15.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.079+0000 7f007a8fd640 10 monclient: _add_conns ranks=[0,1,2,4,3] 2024-03-13T08:11:15.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.079+0000 7f007a8fd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f007406c800 addr v1:172.21.15.132:6789/0 2024-03-13T08:11:15.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.079+0000 7f007a8fd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0074143690 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:15.080 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.079+0000 7f007a8fd640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740af170 0x7f007414e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.079+0000 7f007a8fd640 10 monclient(hunting): picked mon.noname-b con 0x7f00740af170 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.079+0000 7f007a8fd640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00740ae080 con 0x7f007406c800 2024-03-13T08:11:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.079+0000 7f007a8fd640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00740ae360 con 0x7f0074143690 2024-03-13T08:11:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.079+0000 7f007a8fd640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.079+0000 7f007a8fd640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00740b0280 con 0x7f00740af170 2024-03-13T08:11:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.079+0000 7f007a8fd640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.079+0000 7f007a8fd640 10 monclient(hunting): authenticate will time out at 3502.487302s 2024-03-13T08:11:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.079+0000 7f0073fff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f007406c800 0x7f0074142f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:60834/0 (socket says 172.21.15.132:60834) 2024-03-13T08:11:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.079+0000 7f0073fff640 1 -- 172.21.15.132:0/44104846 learned_addr learned my addr 172.21.15.132:0/44104846 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T08:11:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.079+0000 7f0078e9a640 1 --2- 172.21.15.132:0/44104846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740af170 0x7f007414e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.080+0000 7f0078e9a640 10 monclient(hunting): get_auth_request con 0x7f00740af170 auth_method 0 2024-03-13T08:11:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.080+0000 7f0078e9a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.080+0000 7f0078e9a640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.080+0000 7f0078e9a640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.080+0000 7f0078e9a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.080+0000 7f0078e9a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.080+0000 7f0078e9a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0072ffd640 1 -- 172.21.15.132:0/44104846 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977698272 0 0) 0x7f00740ae360 con 0x7f0074143690 2024-03-13T08:11:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0072ffd640 10 monclient(hunting): _negotiate 2024-03-13T08:11:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0072ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0072ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0072ffd640 10 monclient(hunting): my global_id is 19693 2024-03-13T08:11:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0072ffd640 1 -- 172.21.15.132:0/44104846 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f005c003520 con 0x7f0074143690 2024-03-13T08:11:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0072ffd640 1 -- 172.21.15.132:0/44104846 <== mon.0 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 312693052 0 0) 0x7f00740ae080 con 0x7f007406c800 2024-03-13T08:11:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0072ffd640 10 monclient(hunting): _negotiate 2024-03-13T08:11:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0072ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0072ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0072ffd640 10 monclient(hunting): my global_id is 16047 2024-03-13T08:11:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0072ffd640 1 -- 172.21.15.132:0/44104846 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00740ae360 con 0x7f007406c800 2024-03-13T08:11:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0078e9a640 10 monclient(hunting): handle_auth_done global_id 18995 payload 274 2024-03-13T08:11:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0078e9a640 1 -- 172.21.15.132:0/44104846 >> v1:172.21.15.195:6789/0 conn(0x7f0074143690 legacy=0x7f00740aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0078e9a640 1 -- 172.21.15.132:0/44104846 >> v1:172.21.15.132:6789/0 conn(0x7f007406c800 legacy=0x7f0074142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0078e9a640 10 monclient: _finish_hunting 0 2024-03-13T08:11:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0078e9a640 1 monclient: found mon.noname-b 2024-03-13T08:11:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0078e9a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0078e9a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T08:11:15.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.082+0000 7f0078e9a640 1 -- 172.21.15.132:0/44104846 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007406d630 con 0x7f00740af170 2024-03-13T08:11:15.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.084+0000 7f0078e9a640 1 --2- 172.21.15.132:0/44104846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740af170 0x7f007414e080 secure :-1 s=READY pgs=3540 cs=0 l=1 rev1=1 crypto rx=0x7f00740b0400 tx=0x7f006c02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c3079ead467d3ae server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:15.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.086+0000 7f0072ffd640 1 -- 172.21.15.132:0/44104846 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c039070 con 0x7f00740af170 2024-03-13T08:11:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.086+0000 7f0072ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.086+0000 7f0072ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.086+0000 7f0072ffd640 10 monclient: dump: 2024-03-13T08:11:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:15.087 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.086+0000 7f0072ffd640 10 monclient: _finish_auth 0 2024-03-13T08:11:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.086+0000 7f0072ffd640 10 monclient: _check_auth_tickets 2024-03-13T08:11:15.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.086+0000 7f0072ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:15.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.086+0000 7f007a8fd640 5 monclient: authenticate success, global_id 18995 2024-03-13T08:11:15.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.086+0000 7f007a8fd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T08:11:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.088+0000 7f0072ffd640 1 -- 172.21.15.132:0/44104846 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c02cc40 con 0x7f00740af170 2024-03-13T08:11:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.088+0000 7f0072ffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.088+0000 7f0072ffd640 1 -- 172.21.15.132:0/44104846 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c034440 con 0x7f00740af170 2024-03-13T08:11:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.088+0000 7f007a8fd640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.088+0000 7f007a8fd640 4 set_mon_vals no callback set 2024-03-13T08:11:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.088+0000 7f0072ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.088+0000 7f0072ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T08:11:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.088+0000 7f0072ffd640 10 monclient: dump: 2024-03-13T08:11:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:15.090 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.088+0000 7f007a8fd640 10 monclient: shutdown 2024-03-13T08:11:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.088+0000 7f007a8fd640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.088+0000 7f007a8fd640 1 -- 172.21.15.132:0/44104846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740af170 msgr2=0x7f007414e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.088+0000 7f007a8fd640 1 --2- 172.21.15.132:0/44104846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740af170 0x7f007414e080 secure :-1 s=READY pgs=3540 cs=0 l=1 rev1=1 crypto rx=0x7f00740b0400 tx=0x7f006c02be90 comp rx=0 tx=0).stop 2024-03-13T08:11:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.089+0000 7f007a8fd640 1 -- 172.21.15.132:0/44104846 shutdown_connections 2024-03-13T08:11:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.089+0000 7f007a8fd640 1 --2- 172.21.15.132:0/44104846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740af170 0x7f007414e080 unknown :-1 s=CLOSED pgs=3540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.089+0000 7f007a8fd640 1 -- 172.21.15.132:0/44104846 >> 172.21.15.132:0/44104846 conn(0x7f00740b5d70 msgr2=0x7f00740b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:15.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.089+0000 7f007a8fd640 1 -- 172.21.15.132:0/44104846 shutdown_connections 2024-03-13T08:11:15.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.089+0000 7f007a8fd640 1 -- 172.21.15.132:0/44104846 wait complete. 2024-03-13T08:11:15.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.089+0000 7f007a8fd640 10 monclient: build_initial_monmap 2024-03-13T08:11:15.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.089+0000 7f007a8fd640 10 monclient: monmap: 2024-03-13T08:11:15.094 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:15.094 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:15.094 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:15.094 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:15.094 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:15.094 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.094 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:15.095 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:15.095 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:15.095 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 1 librados: starting msgr at 2024-03-13T08:11:15.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 1 librados: starting objecter 2024-03-13T08:11:15.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 1 Processor -- start 2024-03-13T08:11:15.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 1 -- start start 2024-03-13T08:11:15.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 1 librados: setting wanted keys 2024-03-13T08:11:15.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 1 librados: calling monclient init 2024-03-13T08:11:15.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 10 monclient: init 2024-03-13T08:11:15.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:15.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T08:11:15.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f007406c800 0x7f0074144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 10 monclient(hunting): picked mon.noname-a con 0x7f007406c800 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:15.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740af170 0x7f0074145c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 10 monclient(hunting): picked mon.noname-c con 0x7f00740af170 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:15.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0074143690 0x7f0074143bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 10 monclient(hunting): picked mon.noname-b con 0x7f0074143690 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:15.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:15.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00740af5e0 con 0x7f007406c800 2024-03-13T08:11:15.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:15.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007406cc40 con 0x7f0074143690 2024-03-13T08:11:15.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:15.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00740af760 con 0x7f00740af170 2024-03-13T08:11:15.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:15.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.090+0000 7f007a8fd640 10 monclient(hunting): authenticate will time out at 3502.498972s 2024-03-13T08:11:15.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f00737fe640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740af170 0x7f0074145c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f0073fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f007406c800 0x7f0074144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f0078e9a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0074143690 0x7f0074143bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f0073fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f007406c800 0x7f0074144ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51196/0 (socket says 172.21.15.132:51196) 2024-03-13T08:11:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f0073fff640 1 -- 172.21.15.132:0/3223733394 learned_addr learned my addr 172.21.15.132:0/3223733394 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f0073fff640 10 monclient(hunting): get_auth_request con 0x7f007406c800 auth_method 0 2024-03-13T08:11:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f0073fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f0073fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f0073fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f00737fe640 10 monclient(hunting): get_auth_request con 0x7f00740af170 auth_method 0 2024-03-13T08:11:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f00737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f00737fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f00737fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f0078e9a640 10 monclient(hunting): get_auth_request con 0x7f0074143690 auth_method 0 2024-03-13T08:11:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f0078e9a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f0078e9a640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.091+0000 7f0078e9a640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.092+0000 7f0073fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:15.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.092+0000 7f0073fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:15.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.092+0000 7f0073fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:15.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.092+0000 7f00737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:15.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.092+0000 7f00737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:15.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.092+0000 7f00737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:15.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.092+0000 7f0078e9a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:15.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.092+0000 7f0078e9a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:15.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.092+0000 7f0078e9a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:15.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.095+0000 7f00737fe640 10 monclient(hunting): handle_auth_done global_id 18998 payload 824 2024-03-13T08:11:15.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.095+0000 7f00737fe640 1 -- 172.21.15.132:0/3223733394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0074143690 msgr2=0x7f0074143bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.095+0000 7f00737fe640 1 --2- 172.21.15.132:0/3223733394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0074143690 0x7f0074143bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.095+0000 7f00737fe640 1 -- 172.21.15.132:0/3223733394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f007406c800 msgr2=0x7f0074144ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.095+0000 7f00737fe640 1 --2- 172.21.15.132:0/3223733394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f007406c800 0x7f0074144ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.095+0000 7f00737fe640 10 monclient: _finish_hunting 0 2024-03-13T08:11:15.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.095+0000 7f00737fe640 1 monclient: found mon.noname-c 2024-03-13T08:11:15.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.095+0000 7f00737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:15.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.095+0000 7f00737fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T08:11:15.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.095+0000 7f00737fe640 1 -- 172.21.15.132:0/3223733394 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0074135cf0 con 0x7f00740af170 2024-03-13T08:11:15.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.097+0000 7f00737fe640 1 --2- 172.21.15.132:0/3223733394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740af170 0x7f0074145c70 secure :-1 s=READY pgs=3541 cs=0 l=1 rev1=1 crypto rx=0x7f006800c340 tx=0x7f0068001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:15.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.097+0000 7f0070ff9640 10 client.?.objecter ms_handle_connect 0x7f00740af170 2024-03-13T08:11:15.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.097+0000 7f0070ff9640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:15.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f0070ff9640 1 -- 172.21.15.132:0/3223733394 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068017070 con 0x7f00740af170 2024-03-13T08:11:15.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f0070ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:15.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f0070ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T08:11:15.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f0070ff9640 10 monclient: dump: 2024-03-13T08:11:15.103 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:15.103 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:15.103 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.103 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.103 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:15.103 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.103 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:15.103 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:15.103 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:15.103 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f0070ff9640 10 monclient: _finish_auth 0 2024-03-13T08:11:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f0070ff9640 10 monclient: _check_auth_tickets 2024-03-13T08:11:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f0070ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f0070ff9640 1 -- 172.21.15.132:0/3223733394 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00680052d0 con 0x7f00740af170 2024-03-13T08:11:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f007a8fd640 5 monclient: authenticate success, global_id 18998 2024-03-13T08:11:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f007a8fd640 10 monclient: _renew_subs 2024-03-13T08:11:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f007a8fd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f007a8fd640 1 -- 172.21.15.132:0/3223733394 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00741350d0 con 0x7f00740af170 2024-03-13T08:11:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f0070ff9640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f0070ff9640 1 -- 172.21.15.132:0/3223733394 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068003810 con 0x7f00740af170 2024-03-13T08:11:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f0070ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f0070ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T08:11:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f0070ff9640 10 monclient: dump: 2024-03-13T08:11:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:15.105 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.100+0000 7f00727fc640 4 set_mon_vals no callback set 2024-03-13T08:11:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.101+0000 7f007a8fd640 10 client.18998.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.101+0000 7f007a8fd640 10 monclient: _renew_subs 2024-03-13T08:11:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.101+0000 7f007a8fd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.101+0000 7f007a8fd640 1 -- 172.21.15.132:0/3223733394 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0074145220 con 0x7f00740af170 2024-03-13T08:11:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.101+0000 7f007a8fd640 1 librados: init done 2024-03-13T08:11:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.102+0000 7f007a8fd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.102+0000 7f007a8fd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.102+0000 7f007a8fd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.102+0000 7f007a8fd640 1 -- 172.21.15.132:0/3223733394 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007404f000 con 0x7f00740af170 2024-03-13T08:11:15.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.121+0000 7f0070ff9640 1 -- 172.21.15.132:0/3223733394 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f006801a070 con 0x7f00740af170 2024-03-13T08:11:15.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.122+0000 7f0070ff9640 1 --2- 172.21.15.132:0/3223733394 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f006006d5b0 0x7f006006fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.122+0000 7f0073fff640 1 --2- 172.21.15.132:0/3223733394 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f006006d5b0 0x7f006006fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.122+0000 7f0073fff640 10 monclient: get_auth_request con 0x7f006006d5b0 auth_method 0 2024-03-13T08:11:15.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.122+0000 7f0070ff9640 10 client.18998.objecter ms_handle_connect 0x7f006006d5b0 2024-03-13T08:11:15.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.122+0000 7f0073fff640 1 --2- 172.21.15.132:0/3223733394 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f006006d5b0 0x7f006006fa70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f00640011e0 tx=0x7f0064009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:15.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.124+0000 7f0070ff9640 1 -- 172.21.15.132:0/3223733394 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f0068056700 con 0x7f00740af170 2024-03-13T08:11:15.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.124+0000 7f0070ff9640 10 client.18998.objecter ms_dispatch 0x7f0074000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:15.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.124+0000 7f0070ff9640 3 client.18998.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:15.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.124+0000 7f0070ff9640 3 client.18998.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:15.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.124+0000 7f0070ff9640 20 client.18998.objecter dump_active .. 0 homeless 2024-03-13T08:11:15.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.360+0000 7f0070ff9640 1 -- 172.21.15.132:0/3223733394 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f0068021020 con 0x7f00740af170 2024-03-13T08:11:15.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.360+0000 7f0070ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:15.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.360+0000 7f0070ff9640 10 monclient: _finish_command 1 = system:0 2024-03-13T08:11:15.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.539+0000 7f007a8fd640 1 -- 172.21.15.132:0/3223733394 --> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f00740b4d20 con 0x7f006006d5b0 2024-03-13T08:11:15.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.539+0000 7f0070ff9640 1 -- 172.21.15.132:0/3223733394 <== mgr.4104 v2:172.21.15.132:6800/3847550149 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f00740b4d20 con 0x7f006006d5b0 2024-03-13T08:11:15.539 INFO:teuthology.orchestra.run.smithi132.stderr:instructed osd(s) 3 to deep-scrub 2024-03-13T08:11:15.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 10 librados: watch_flush enter 2024-03-13T08:11:15.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 10 librados: watch_flush exit 2024-03-13T08:11:15.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 20 client.18998.objecter shutdown clearing up homeless session... 2024-03-13T08:11:15.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 10 client.18998.objecter successfully canceled tick 2024-03-13T08:11:15.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 1 -- 172.21.15.132:0/3223733394 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f006006d5b0 msgr2=0x7f006006fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 1 --2- 172.21.15.132:0/3223733394 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f006006d5b0 0x7f006006fa70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f00640011e0 tx=0x7f0064009040 comp rx=0 tx=0).stop 2024-03-13T08:11:15.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 10 monclient: shutdown 2024-03-13T08:11:15.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:15.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 1 -- 172.21.15.132:0/3223733394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740af170 msgr2=0x7f0074145c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 1 --2- 172.21.15.132:0/3223733394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740af170 0x7f0074145c70 secure :-1 s=READY pgs=3541 cs=0 l=1 rev1=1 crypto rx=0x7f006800c340 tx=0x7f0068001e70 comp rx=0 tx=0).stop 2024-03-13T08:11:15.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 1 -- 172.21.15.132:0/3223733394 shutdown_connections 2024-03-13T08:11:15.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 1 --2- 172.21.15.132:0/3223733394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0074143690 0x7f0074143bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 1 --2- 172.21.15.132:0/3223733394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740af170 0x7f0074145c70 unknown :-1 s=CLOSED pgs=3541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 1 --2- 172.21.15.132:0/3223733394 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f006006d5b0 0x7f006006fa70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 1 --2- 172.21.15.132:0/3223733394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f007406c800 0x7f0074144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 1 -- 172.21.15.132:0/3223733394 >> 172.21.15.132:0/3223733394 conn(0x7f00740b5d70 msgr2=0x7f0074142f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:15.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 1 -- 172.21.15.132:0/3223733394 shutdown_connections 2024-03-13T08:11:15.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 1 -- 172.21.15.132:0/3223733394 wait complete. 2024-03-13T08:11:15.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.541+0000 7f007a8fd640 1 librados: shutdown 2024-03-13T08:11:15.554 INFO:tasks.ceph:Scrubbing osd.4 2024-03-13T08:11:15.554 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-13T08:11:15.658 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.658+0000 7fca81b74640 10 monclient: get_monmap_and_config 2024-03-13T08:11:15.658 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.658+0000 7fca81b74640 10 monclient: build_initial_monmap 2024-03-13T08:11:15.658 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.658+0000 7fca81b74640 10 monclient: monmap: 2024-03-13T08:11:15.658 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:15.659 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:15.659 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:15.658978+0000 2024-03-13T08:11:15.659 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:15.658978+0000 2024-03-13T08:11:15.659 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:15.659 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.659 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:15.659 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:15.659 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:15.659 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:15.659 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:15.659 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.658+0000 7fca81b74640 1 Processor -- start 2024-03-13T08:11:15.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 1 -- start start 2024-03-13T08:11:15.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 10 monclient: init 2024-03-13T08:11:15.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:15.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 10 monclient: _add_conns ranks=[2,4,3,1,0] 2024-03-13T08:11:15.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca7c0b8760 0x7fca7c1426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 10 monclient(hunting): picked mon.noname-b con 0x7fca7c0b8760 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:15.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fca7c0b8b40 0x7fca7c0b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 10 monclient(hunting): picked mon.noname-c con 0x7fca7c0b8b40 addr v2:172.21.15.195:3300/0 2024-03-13T08:11:15.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fca7c0b24e0 0x7fca7c0aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 10 monclient(hunting): picked mon.noname-a con 0x7fca7c0b24e0 addr v2:172.21.15.132:3300/0 2024-03-13T08:11:15.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:15.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca7c0b1b00 con 0x7fca7c0b24e0 2024-03-13T08:11:15.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:15.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca7c0b1400 con 0x7fca7c0b8b40 2024-03-13T08:11:15.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:15.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0b16e0 con 0x7fca7c0b8760 2024-03-13T08:11:15.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:15.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.659+0000 7fca81b74640 10 monclient(hunting): authenticate will time out at 3503.067855s 2024-03-13T08:11:15.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7bfff640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fca7c0b24e0 0x7fca7c0aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7b7fe640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca7c0b8760 0x7fca7c1426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7affd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fca7c0b8b40 0x7fca7c0b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7bfff640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fca7c0b24e0 0x7fca7c0aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56118/0 (socket says 172.21.15.132:56118) 2024-03-13T08:11:15.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7bfff640 1 -- 172.21.15.132:0/369712307 learned_addr learned my addr 172.21.15.132:0/369712307 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:15.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7bfff640 10 monclient(hunting): get_auth_request con 0x7fca7c0b24e0 auth_method 0 2024-03-13T08:11:15.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:15.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7bfff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7affd640 10 monclient(hunting): get_auth_request con 0x7fca7c0b8b40 auth_method 0 2024-03-13T08:11:15.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:15.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7affd640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7affd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7b7fe640 10 monclient(hunting): get_auth_request con 0x7fca7c0b8760 auth_method 0 2024-03-13T08:11:15.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:15.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7b7fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.660+0000 7fca7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.661+0000 7fca7affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:15.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.661+0000 7fca7affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:15.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.661+0000 7fca7affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:15.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.661+0000 7fca7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:15.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.661+0000 7fca7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:15.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.661+0000 7fca7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:15.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.661+0000 7fca7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:15.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.661+0000 7fca7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:15.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.661+0000 7fca7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:15.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.663+0000 7fca7affd640 10 monclient(hunting): handle_auth_done global_id 19699 payload 274 2024-03-13T08:11:15.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.663+0000 7fca7affd640 1 -- 172.21.15.132:0/369712307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca7c0b8760 msgr2=0x7fca7c1426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:15.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.663+0000 7fca7affd640 1 --2- 172.21.15.132:0/369712307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca7c0b8760 0x7fca7c1426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.663+0000 7fca7affd640 1 -- 172.21.15.132:0/369712307 >> v2:172.21.15.132:3300/0 conn(0x7fca7c0b24e0 msgr2=0x7fca7c0aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:15.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.663+0000 7fca7affd640 1 --2- 172.21.15.132:0/369712307 >> v2:172.21.15.132:3300/0 conn(0x7fca7c0b24e0 0x7fca7c0aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.663+0000 7fca7affd640 10 monclient: _finish_hunting 0 2024-03-13T08:11:15.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.663+0000 7fca7affd640 1 monclient: found mon.noname-c 2024-03-13T08:11:15.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.663+0000 7fca7affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:15.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.663+0000 7fca7affd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T08:11:15.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.663+0000 7fca7affd640 1 -- 172.21.15.132:0/369712307 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c0b4840 con 0x7fca7c0b8b40 2024-03-13T08:11:15.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.665+0000 7fca7affd640 1 --2- 172.21.15.132:0/369712307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca7c0b8b40 0x7fca7c0b1fa0 secure :-1 s=READY pgs=4734 cs=0 l=1 rev1=1 crypto rx=0x7fca7c06ae10 tx=0x7fca7002b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ff114c67952ed7a server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:15.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca7a7fc640 1 -- 172.21.15.132:0/369712307 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70038070 con 0x7fca7c0b8b40 2024-03-13T08:11:15.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:15.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca7a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T08:11:15.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca7a7fc640 10 monclient: dump: 2024-03-13T08:11:15.669 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:15.669 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:15.669 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.669 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.669 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:15.669 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.670 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:15.670 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:15.670 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:15.670 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca7a7fc640 10 monclient: _finish_auth 0 2024-03-13T08:11:15.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca7a7fc640 10 monclient: _check_auth_tickets 2024-03-13T08:11:15.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:15.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca81b74640 5 monclient: authenticate success, global_id 19699 2024-03-13T08:11:15.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca81b74640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T08:11:15.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca7a7fc640 1 -- 172.21.15.132:0/369712307 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca70031700 con 0x7fca7c0b8b40 2024-03-13T08:11:15.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca7a7fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:15.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca81b74640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:15.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca81b74640 4 set_mon_vals no callback set 2024-03-13T08:11:15.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca7a7fc640 1 -- 172.21.15.132:0/369712307 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70031ad0 con 0x7fca7c0b8b40 2024-03-13T08:11:15.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:15.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca7a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:15.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.668+0000 7fca7a7fc640 10 monclient: dump: 2024-03-13T08:11:15.671 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:15.671 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:15.671 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.671 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.671 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:15.671 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.671 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:15.671 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:15.671 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:15.672 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.669+0000 7fca81b74640 10 monclient: shutdown 2024-03-13T08:11:15.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.669+0000 7fca81b74640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:15.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.669+0000 7fca81b74640 1 -- 172.21.15.132:0/369712307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca7c0b8b40 msgr2=0x7fca7c0b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.669+0000 7fca81b74640 1 --2- 172.21.15.132:0/369712307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca7c0b8b40 0x7fca7c0b1fa0 secure :-1 s=READY pgs=4734 cs=0 l=1 rev1=1 crypto rx=0x7fca7c06ae10 tx=0x7fca7002b7a0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.669+0000 7fca81b74640 1 -- 172.21.15.132:0/369712307 shutdown_connections 2024-03-13T08:11:15.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.669+0000 7fca81b74640 1 --2- 172.21.15.132:0/369712307 >> v2:172.21.15.132:3300/0 conn(0x7fca7c0b24e0 0x7fca7c0aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.669+0000 7fca81b74640 1 --2- 172.21.15.132:0/369712307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca7c0b8b40 0x7fca7c0b1fa0 unknown :-1 s=CLOSED pgs=4734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.669+0000 7fca81b74640 1 --2- 172.21.15.132:0/369712307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca7c0b8760 0x7fca7c1426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.669+0000 7fca81b74640 1 -- 172.21.15.132:0/369712307 >> 172.21.15.132:0/369712307 conn(0x7fca7c06d4f0 msgr2=0x7fca7c0b8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:15.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.669+0000 7fca81b74640 1 -- 172.21.15.132:0/369712307 shutdown_connections 2024-03-13T08:11:15.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.669+0000 7fca81b74640 1 -- 172.21.15.132:0/369712307 wait complete. 2024-03-13T08:11:15.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.670+0000 7fca81b74640 10 monclient: build_initial_monmap 2024-03-13T08:11:15.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.670+0000 7fca81b74640 10 monclient: monmap: 2024-03-13T08:11:15.677 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:15.677 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:15.677 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:15.677 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:15.677 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:15.677 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.677 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:15.678 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:15.678 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:15.678 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.670+0000 7fca81b74640 1 librados: starting msgr at 2024-03-13T08:11:15.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.670+0000 7fca81b74640 1 librados: starting objecter 2024-03-13T08:11:15.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.670+0000 7fca81b74640 1 Processor -- start 2024-03-13T08:11:15.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.670+0000 7fca81b74640 1 -- start start 2024-03-13T08:11:15.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.670+0000 7fca81b74640 1 librados: setting wanted keys 2024-03-13T08:11:15.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.670+0000 7fca81b74640 1 librados: calling monclient init 2024-03-13T08:11:15.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.670+0000 7fca81b74640 10 monclient: init 2024-03-13T08:11:15.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.671+0000 7fca81b74640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:15.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.671+0000 7fca81b74640 10 monclient: _add_conns ranks=[2,1,0] 2024-03-13T08:11:15.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.671+0000 7fca81b74640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca7c0b8760 0x7fca7c1c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.671+0000 7fca81b74640 10 monclient(hunting): picked mon.noname-c con 0x7fca7c0b8760 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:15.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.671+0000 7fca81b74640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca7c0ab070 0x7fca7c0ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.671+0000 7fca81b74640 10 monclient(hunting): picked mon.noname-b con 0x7fca7c0ab070 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:15.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.671+0000 7fca81b74640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca7c13a550 0x7fca7c13a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.671+0000 7fca81b74640 10 monclient(hunting): picked mon.noname-a con 0x7fca7c13a550 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:15.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.671+0000 7fca81b74640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:15.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.671+0000 7fca81b74640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0aef40 con 0x7fca7c13a550 2024-03-13T08:11:15.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.671+0000 7fca81b74640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:15.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.671+0000 7fca81b74640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0b0150 con 0x7fca7c0ab070 2024-03-13T08:11:15.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.671+0000 7fca81b74640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:15.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.671+0000 7fca81b74640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0af0c0 con 0x7fca7c0b8760 2024-03-13T08:11:15.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.671+0000 7fca81b74640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:15.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.671+0000 7fca81b74640 10 monclient(hunting): authenticate will time out at 3503.079632s 2024-03-13T08:11:15.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca7c13a550 0x7fca7c13a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7b7fe640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca7c0b8760 0x7fca7c1c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7affd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca7c0ab070 0x7fca7c0ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca7c13a550 0x7fca7c13a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56128/0 (socket says 172.21.15.132:56128) 2024-03-13T08:11:15.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7bfff640 1 -- 172.21.15.132:0/1156517655 learned_addr learned my addr 172.21.15.132:0/1156517655 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:15.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7bfff640 10 monclient(hunting): get_auth_request con 0x7fca7c13a550 auth_method 0 2024-03-13T08:11:15.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:15.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7bfff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7b7fe640 10 monclient(hunting): get_auth_request con 0x7fca7c0b8760 auth_method 0 2024-03-13T08:11:15.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:15.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7b7fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7affd640 10 monclient(hunting): get_auth_request con 0x7fca7c0ab070 auth_method 0 2024-03-13T08:11:15.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:15.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7affd640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7affd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:15.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:15.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:15.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:15.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:15.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:15.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:15.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:15.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.672+0000 7fca7affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:15.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.675+0000 7fca7affd640 10 monclient(hunting): handle_auth_done global_id 19702 payload 824 2024-03-13T08:11:15.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.675+0000 7fca7affd640 1 -- 172.21.15.132:0/1156517655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca7c0b8760 msgr2=0x7fca7c1c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.675+0000 7fca7affd640 1 --2- 172.21.15.132:0/1156517655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca7c0b8760 0x7fca7c1c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.675+0000 7fca7affd640 1 -- 172.21.15.132:0/1156517655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca7c13a550 msgr2=0x7fca7c13a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.675+0000 7fca7affd640 1 --2- 172.21.15.132:0/1156517655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca7c13a550 0x7fca7c13a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.675+0000 7fca7affd640 10 monclient: _finish_hunting 0 2024-03-13T08:11:15.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.675+0000 7fca7affd640 1 monclient: found mon.noname-b 2024-03-13T08:11:15.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.675+0000 7fca7affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:15.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.675+0000 7fca7affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T08:11:15.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.675+0000 7fca7affd640 1 -- 172.21.15.132:0/1156517655 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c1c9ca0 con 0x7fca7c0ab070 2024-03-13T08:11:15.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.678+0000 7fca7affd640 1 --2- 172.21.15.132:0/1156517655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca7c0ab070 0x7fca7c0ab4f0 secure :-1 s=READY pgs=4735 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0b1580 tx=0x7fca700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:15.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.678+0000 7fca63fff640 10 client.?.objecter ms_handle_connect 0x7fca7c0ab070 2024-03-13T08:11:15.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.678+0000 7fca63fff640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:15.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca63fff640 1 -- 172.21.15.132:0/1156517655 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70041020 con 0x7fca7c0ab070 2024-03-13T08:11:15.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca63fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:15.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca63fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:15.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca63fff640 10 monclient: dump: 2024-03-13T08:11:15.687 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:15.688 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:15.688 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.688 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.688 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:15.688 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.688 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:15.688 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:15.688 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:15.688 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca63fff640 10 monclient: _finish_auth 0 2024-03-13T08:11:15.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca63fff640 10 monclient: _check_auth_tickets 2024-03-13T08:11:15.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca63fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:15.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca63fff640 1 -- 172.21.15.132:0/1156517655 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca700337a0 con 0x7fca7c0ab070 2024-03-13T08:11:15.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca81b74640 5 monclient: authenticate success, global_id 19702 2024-03-13T08:11:15.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca81b74640 10 monclient: _renew_subs 2024-03-13T08:11:15.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca81b74640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:15.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca81b74640 1 -- 172.21.15.132:0/1156517655 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7c1c8f80 con 0x7fca7c0ab070 2024-03-13T08:11:15.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca63fff640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:15.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca63fff640 1 -- 172.21.15.132:0/1156517655 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70038040 con 0x7fca7c0ab070 2024-03-13T08:11:15.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca63fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:15.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca63fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:15.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca63fff640 10 monclient: dump: 2024-03-13T08:11:15.689 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:15.689 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:15.689 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.690 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.690 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:15.690 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.690 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:15.690 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:15.690 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:15.690 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca79ffb640 4 set_mon_vals no callback set 2024-03-13T08:11:15.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca81b74640 10 client.19702.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:15.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca81b74640 10 monclient: _renew_subs 2024-03-13T08:11:15.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca81b74640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:15.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca81b74640 1 -- 172.21.15.132:0/1156517655 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7c1ca1f0 con 0x7fca7c0ab070 2024-03-13T08:11:15.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.681+0000 7fca81b74640 1 librados: init done 2024-03-13T08:11:15.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.682+0000 7fca81b74640 10 client.19702.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:15.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.682+0000 7fca81b74640 15 client.19702.objecter _session_command_op_assign -1 1 2024-03-13T08:11:15.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.682+0000 7fca81b74640 20 client.19702.objecter _get_session osd=-1 returning homeless 2024-03-13T08:11:15.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.682+0000 7fca81b74640 10 client.19702.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:15.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.682+0000 7fca81b74640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:15.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.682+0000 7fca81b74640 1 -- 172.21.15.132:0/1156517655 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca7c04f000 con 0x7fca7c0ab070 2024-03-13T08:11:15.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.701+0000 7fca63fff640 1 -- 172.21.15.132:0/1156517655 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fca70044020 con 0x7fca7c0ab070 2024-03-13T08:11:15.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.701+0000 7fca63fff640 1 --2- 172.21.15.132:0/1156517655 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fca6806d5b0 0x7fca6806fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.702+0000 7fca7b7fe640 1 --2- 172.21.15.132:0/1156517655 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fca6806d5b0 0x7fca6806fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.702+0000 7fca7b7fe640 10 monclient: get_auth_request con 0x7fca6806d5b0 auth_method 0 2024-03-13T08:11:15.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.702+0000 7fca63fff640 10 client.19702.objecter ms_handle_connect 0x7fca6806d5b0 2024-03-13T08:11:15.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.702+0000 7fca7b7fe640 1 --2- 172.21.15.132:0/1156517655 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fca6806d5b0 0x7fca6806fa70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009ac0 tx=0x7fca6c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:15.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 1 -- 172.21.15.132:0/1156517655 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7fca700787e0 con 0x7fca7c0ab070 2024-03-13T08:11:15.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 10 client.19702.objecter ms_dispatch 0x7fca7c000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:15.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 3 client.19702.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:15.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 3 client.19702.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:15.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 10 client.19702.objecter checking command 1 2024-03-13T08:11:15.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 1 --2- 172.21.15.132:0/1156517655 >> [v2:172.21.15.195:6802/699638467,v1:172.21.15.195:6803/699638467] conn(0x7fca68075220 0x7fca680776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 20 client.19702.objecter _get_session s=0x7fca68074b50 osd=4 3 2024-03-13T08:11:15.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 20 client.19702.objecter put_session s=0x7fca68074b50 osd=4 3 2024-03-13T08:11:15.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 15 client.19702.objecter _session_command_op_remove -1 1 2024-03-13T08:11:15.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 20 client.19702.objecter _get_session s=0x7fca68074b50 osd=4 3 2024-03-13T08:11:15.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 20 client.19702.objecter get_session s=0x7fca68074b50 osd=4 3 2024-03-13T08:11:15.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 15 client.19702.objecter _session_command_op_assign 4 1 2024-03-13T08:11:15.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 20 client.19702.objecter put_session s=0x7fca68074b50 osd=4 4 2024-03-13T08:11:15.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 10 client.19702.objecter _send_command 1 2024-03-13T08:11:15.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 1 -- 172.21.15.132:0/1156517655 --> [v2:172.21.15.195:6802/699638467,v1:172.21.15.195:6803/699638467] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca700067f0 con 0x7fca68075220 2024-03-13T08:11:15.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 20 client.19702.objecter dump_active .. 0 homeless 2024-03-13T08:11:15.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 1 -- 172.21.15.132:0/1156517655 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=1293) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca700769e0 con 0x7fca7c0ab070 2024-03-13T08:11:15.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca63fff640 10 monclient: handle_get_version_reply finishing 1 version 1293 2024-03-13T08:11:15.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.704+0000 7fca7bfff640 1 --2- 172.21.15.132:0/1156517655 >> [v2:172.21.15.195:6802/699638467,v1:172.21.15.195:6803/699638467] conn(0x7fca68075220 0x7fca680776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.705+0000 7fca7bfff640 10 monclient: get_auth_request con 0x7fca68075220 auth_method 0 2024-03-13T08:11:15.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.717+0000 7fca7bfff640 1 --2- 172.21.15.132:0/1156517655 >> [v2:172.21.15.195:6802/699638467,v1:172.21.15.195:6803/699638467] conn(0x7fca68075220 0x7fca680776c0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:15.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.717+0000 7fca63fff640 10 client.19702.objecter ms_handle_connect 0x7fca68075220 2024-03-13T08:11:15.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.741+0000 7fca63fff640 1 -- 172.21.15.132:0/1156517655 <== osd.4 v2:172.21.15.195:6802/699638467 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fca700067f0 con 0x7fca68075220 2024-03-13T08:11:15.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.741+0000 7fca63fff640 10 client.19702.objecter ms_dispatch 0x7fca7c000b90 command_reply(tid 1: 0 ) v1 2024-03-13T08:11:15.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.741+0000 7fca63fff640 10 client.19702.objecter _finish_command 1 = system:0 2024-03-13T08:11:15.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.741+0000 7fca63fff640 20 client.19702.objecter put_session s=0x7fca68074b50 osd=4 4 2024-03-13T08:11:15.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.741+0000 7fca63fff640 15 client.19702.objecter _session_command_op_remove 4 1 2024-03-13T08:11:15.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.753+0000 7fca81b74640 10 client.19702.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-03-13T08:11:15.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.753+0000 7fca81b74640 15 client.19702.objecter _session_command_op_assign -1 2 2024-03-13T08:11:15.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.753+0000 7fca81b74640 20 client.19702.objecter _get_session s=0x7fca68074b50 osd=4 3 2024-03-13T08:11:15.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.753+0000 7fca81b74640 20 client.19702.objecter put_session s=0x7fca68074b50 osd=4 3 2024-03-13T08:11:15.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.753+0000 7fca81b74640 20 client.19702.objecter _get_session s=0x7fca68074b50 osd=4 3 2024-03-13T08:11:15.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.753+0000 7fca81b74640 15 client.19702.objecter _session_command_op_remove -1 2 2024-03-13T08:11:15.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.753+0000 7fca81b74640 20 client.19702.objecter get_session s=0x7fca68074b50 osd=4 3 2024-03-13T08:11:15.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.753+0000 7fca81b74640 15 client.19702.objecter _session_command_op_assign 4 2 2024-03-13T08:11:15.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.753+0000 7fca81b74640 20 client.19702.objecter put_session s=0x7fca68074b50 osd=4 4 2024-03-13T08:11:15.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.753+0000 7fca81b74640 10 client.19702.objecter _send_command 2 2024-03-13T08:11:15.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.753+0000 7fca81b74640 1 -- 172.21.15.132:0/1156517655 --> [v2:172.21.15.195:6802/699638467,v1:172.21.15.195:6803/699638467] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fca7c13bd40 con 0x7fca68075220 2024-03-13T08:11:15.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca63fff640 1 -- 172.21.15.132:0/1156517655 <== osd.4 v2:172.21.15.195:6802/699638467 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fca7c13bd40 con 0x7fca68075220 2024-03-13T08:11:15.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca63fff640 10 client.19702.objecter ms_dispatch 0x7fca7c000b90 command_reply(tid 2: 0 ) v1 2024-03-13T08:11:15.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca63fff640 10 client.19702.objecter _finish_command 2 = system:0 2024-03-13T08:11:15.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca63fff640 20 client.19702.objecter put_session s=0x7fca68074b50 osd=4 4 2024-03-13T08:11:15.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca63fff640 15 client.19702.objecter _session_command_op_remove 4 2 2024-03-13T08:11:15.770 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-13T08:11:15.770 INFO:teuthology.orchestra.run.smithi132.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-13T08:11:15.770 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-13T08:11:15.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 10 librados: watch_flush enter 2024-03-13T08:11:15.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 10 librados: watch_flush exit 2024-03-13T08:11:15.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 10 client.19702.objecter close_session for osd.4 2024-03-13T08:11:15.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 1 -- 172.21.15.132:0/1156517655 >> [v2:172.21.15.195:6802/699638467,v1:172.21.15.195:6803/699638467] conn(0x7fca68075220 msgr2=0x7fca680776c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 1 --2- 172.21.15.132:0/1156517655 >> [v2:172.21.15.195:6802/699638467,v1:172.21.15.195:6803/699638467] conn(0x7fca68075220 0x7fca680776c0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 20 client.19702.objecter put_session s=0x7fca68074b50 osd=4 1 2024-03-13T08:11:15.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 20 client.19702.objecter shutdown clearing up homeless session... 2024-03-13T08:11:15.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 10 client.19702.objecter successfully canceled tick 2024-03-13T08:11:15.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 1 -- 172.21.15.132:0/1156517655 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fca6806d5b0 msgr2=0x7fca6806fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 1 --2- 172.21.15.132:0/1156517655 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fca6806d5b0 0x7fca6806fa70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009ac0 tx=0x7fca6c009190 comp rx=0 tx=0).stop 2024-03-13T08:11:15.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 10 monclient: shutdown 2024-03-13T08:11:15.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:15.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 1 -- 172.21.15.132:0/1156517655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca7c0ab070 msgr2=0x7fca7c0ab4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 1 --2- 172.21.15.132:0/1156517655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca7c0ab070 0x7fca7c0ab4f0 secure :-1 s=READY pgs=4735 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0b1580 tx=0x7fca700046c0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca7bfff640 1 -- 172.21.15.132:0/1156517655 reap_dead start 2024-03-13T08:11:15.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 1 -- 172.21.15.132:0/1156517655 shutdown_connections 2024-03-13T08:11:15.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 1 -- 172.21.15.132:0/1156517655 >> 172.21.15.132:0/1156517655 conn(0x7fca7c06d4f0 msgr2=0x7fca7c0b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:15.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 1 -- 172.21.15.132:0/1156517655 shutdown_connections 2024-03-13T08:11:15.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 1 -- 172.21.15.132:0/1156517655 wait complete. 2024-03-13T08:11:15.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.769+0000 7fca81b74640 1 librados: shutdown 2024-03-13T08:11:15.780 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-13T08:11:15.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.881+0000 7fa92e645640 10 monclient: get_monmap_and_config 2024-03-13T08:11:15.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.881+0000 7fa92e645640 10 monclient: build_initial_monmap 2024-03-13T08:11:15.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.881+0000 7fa92e645640 10 monclient: monmap: 2024-03-13T08:11:15.882 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:15.882 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:15.882 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:15.882549+0000 2024-03-13T08:11:15.882 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:15.882549+0000 2024-03-13T08:11:15.882 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:15.882 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.882 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:15.883 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:15.883 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:15.883 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:15.883 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:15.883 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.882+0000 7fa92e645640 1 Processor -- start 2024-03-13T08:11:15.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.882+0000 7fa92e645640 1 -- start start 2024-03-13T08:11:15.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.882+0000 7fa92e645640 10 monclient: init 2024-03-13T08:11:15.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa92e645640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:15.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa92e645640 10 monclient: _add_conns ranks=[2,3,1,4,0] 2024-03-13T08:11:15.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa92e645640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa928070820 0x7fa9280ae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa92e645640 10 monclient(hunting): picked mon.noname-b con 0x7fa928070820 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:15.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa92e645640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa928070c00 0x7fa928144150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa92e645640 10 monclient(hunting): picked mon.noname-a con 0x7fa928070c00 addr v2:172.21.15.132:3300/0 2024-03-13T08:11:15.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa92e645640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa92813d600 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:15.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa92e645640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa928072870 con 0x7fa92813d600 2024-03-13T08:11:15.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa92e645640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:15.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa92e645640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9280ae0d0 con 0x7fa928070c00 2024-03-13T08:11:15.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa92e645640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:15.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa92e645640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280ae320 con 0x7fa928070820 2024-03-13T08:11:15.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa92e645640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:15.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa92e645640 10 monclient(hunting): authenticate will time out at 3503.291285s 2024-03-13T08:11:15.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa927fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa928070820 0x7fa9280ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa9277fe640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa928070c00 0x7fa928144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa92cbe2640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fa92813d600 0x7fa92813da90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.132:35264/0 (socket says 172.21.15.132:35264) 2024-03-13T08:11:15.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.883+0000 7fa92cbe2640 1 -- 172.21.15.132:0/2181881384 learned_addr learned my addr 172.21.15.132:0/2181881384 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T08:11:15.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.884+0000 7fa9277fe640 10 monclient(hunting): get_auth_request con 0x7fa928070c00 auth_method 0 2024-03-13T08:11:15.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.884+0000 7fa9277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:15.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.884+0000 7fa9277fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.886 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.884+0000 7fa9277fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.884+0000 7fa927fff640 10 monclient(hunting): get_auth_request con 0x7fa928070820 auth_method 0 2024-03-13T08:11:15.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.884+0000 7fa927fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:15.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.884+0000 7fa927fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.884+0000 7fa927fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.884+0000 7fa9277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:15.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.884+0000 7fa9277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:15.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.884+0000 7fa9277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:15.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.884+0000 7fa927fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:15.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.884+0000 7fa927fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:15.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.884+0000 7fa927fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:15.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.886+0000 7fa926ffd640 1 -- 172.21.15.132:0/2181881384 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892801984 0 0) 0x7fa928072870 con 0x7fa92813d600 2024-03-13T08:11:15.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.886+0000 7fa926ffd640 10 monclient(hunting): _negotiate 2024-03-13T08:11:15.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.886+0000 7fa926ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.886+0000 7fa926ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.886+0000 7fa926ffd640 10 monclient(hunting): my global_id is 19705 2024-03-13T08:11:15.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.886+0000 7fa926ffd640 1 -- 172.21.15.132:0/2181881384 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa910003520 con 0x7fa92813d600 2024-03-13T08:11:15.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.887+0000 7fa9277fe640 10 monclient(hunting): handle_auth_done global_id 16059 payload 274 2024-03-13T08:11:15.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.887+0000 7fa9277fe640 1 -- 172.21.15.132:0/2181881384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa928070820 msgr2=0x7fa9280ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:15.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.887+0000 7fa9277fe640 1 --2- 172.21.15.132:0/2181881384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa928070820 0x7fa9280ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.887+0000 7fa9277fe640 1 -- 172.21.15.132:0/2181881384 >> v1:172.21.15.195:6789/0 conn(0x7fa92813d600 legacy=0x7fa92813da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.887+0000 7fa9277fe640 10 monclient: _finish_hunting 0 2024-03-13T08:11:15.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.887+0000 7fa9277fe640 1 monclient: found mon.noname-a 2024-03-13T08:11:15.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.887+0000 7fa9277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:15.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.887+0000 7fa9277fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T08:11:15.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.887+0000 7fa9277fe640 1 -- 172.21.15.132:0/2181881384 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa928072ae0 con 0x7fa928070c00 2024-03-13T08:11:15.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.889+0000 7fa9277fe640 1 --2- 172.21.15.132:0/2181881384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa928070c00 0x7fa928144150 secure :-1 s=READY pgs=3268 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0097a0 tx=0x7fa91c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0661caa7e32ef0a server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:15.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa926ffd640 1 -- 172.21.15.132:0/2181881384 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0315a0 con 0x7fa928070c00 2024-03-13T08:11:15.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa926ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:15.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa926ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T08:11:15.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa926ffd640 10 monclient: dump: 2024-03-13T08:11:15.894 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:15.894 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:15.894 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.894 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.894 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:15.894 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.894 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:15.894 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:15.894 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:15.894 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa926ffd640 10 monclient: _finish_auth 0 2024-03-13T08:11:15.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa926ffd640 10 monclient: _check_auth_tickets 2024-03-13T08:11:15.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa926ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:15.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa926ffd640 1 -- 172.21.15.132:0/2181881384 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91c037070 con 0x7fa928070c00 2024-03-13T08:11:15.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa92e645640 5 monclient: authenticate success, global_id 16059 2024-03-13T08:11:15.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa92e645640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T08:11:15.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa926ffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:15.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa92e645640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:15.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa926ffd640 1 -- 172.21.15.132:0/2181881384 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c031960 con 0x7fa928070c00 2024-03-13T08:11:15.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa92e645640 4 set_mon_vals no callback set 2024-03-13T08:11:15.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa926ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:15.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa926ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T08:11:15.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa926ffd640 10 monclient: dump: 2024-03-13T08:11:15.895 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:15.895 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:15.896 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.896 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.896 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:15.896 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.896 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:15.896 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:15.896 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:15.896 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa92e645640 10 monclient: shutdown 2024-03-13T08:11:15.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa92e645640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:15.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa92e645640 1 -- 172.21.15.132:0/2181881384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa928070c00 msgr2=0x7fa928144150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa92e645640 1 --2- 172.21.15.132:0/2181881384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa928070c00 0x7fa928144150 secure :-1 s=READY pgs=3268 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0097a0 tx=0x7fa91c02b840 comp rx=0 tx=0).stop 2024-03-13T08:11:15.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa92e645640 1 -- 172.21.15.132:0/2181881384 shutdown_connections 2024-03-13T08:11:15.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa92e645640 1 --2- 172.21.15.132:0/2181881384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa928070c00 0x7fa928144150 unknown :-1 s=CLOSED pgs=3268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa92e645640 1 --2- 172.21.15.132:0/2181881384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa928070820 0x7fa9280ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.893+0000 7fa92e645640 1 -- 172.21.15.132:0/2181881384 >> 172.21.15.132:0/2181881384 conn(0x7fa9280b3e20 msgr2=0x7fa9280b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:15.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.894+0000 7fa92e645640 1 -- 172.21.15.132:0/2181881384 shutdown_connections 2024-03-13T08:11:15.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.894+0000 7fa92e645640 1 -- 172.21.15.132:0/2181881384 wait complete. 2024-03-13T08:11:15.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.894+0000 7fa92e645640 10 monclient: build_initial_monmap 2024-03-13T08:11:15.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.894+0000 7fa92e645640 10 monclient: monmap: 2024-03-13T08:11:15.901 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:15.901 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:15.901 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:15.901 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:15.901 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:15.901 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.901 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:15.901 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:15.901 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:15.901 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.895+0000 7fa92e645640 1 librados: starting msgr at 2024-03-13T08:11:15.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.895+0000 7fa92e645640 1 librados: starting objecter 2024-03-13T08:11:15.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.895+0000 7fa92e645640 1 Processor -- start 2024-03-13T08:11:15.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.895+0000 7fa92e645640 1 -- start start 2024-03-13T08:11:15.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.895+0000 7fa92e645640 1 librados: setting wanted keys 2024-03-13T08:11:15.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.895+0000 7fa92e645640 1 librados: calling monclient init 2024-03-13T08:11:15.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.895+0000 7fa92e645640 10 monclient: init 2024-03-13T08:11:15.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.895+0000 7fa92e645640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:15.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.895+0000 7fa92e645640 10 monclient: _add_conns ranks=[2,1,0] 2024-03-13T08:11:15.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.895+0000 7fa92e645640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa928070820 0x7fa928137e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.895+0000 7fa92e645640 10 monclient(hunting): picked mon.noname-c con 0x7fa928070820 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:15.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92e645640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa928070c00 0x7fa9281340a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92e645640 10 monclient(hunting): picked mon.noname-b con 0x7fa928070c00 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:15.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92e645640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa92813d600 0x7fa9281d9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92e645640 10 monclient(hunting): picked mon.noname-a con 0x7fa92813d600 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:15.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92e645640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:15.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92e645640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa928144810 con 0x7fa92813d600 2024-03-13T08:11:15.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92e645640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:15.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92e645640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa928144690 con 0x7fa928070c00 2024-03-13T08:11:15.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92e645640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:15.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92e645640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92814acf0 con 0x7fa928070820 2024-03-13T08:11:15.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92e645640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:15.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92e645640 10 monclient(hunting): authenticate will time out at 3503.304145s 2024-03-13T08:11:15.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92cbe2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa92813d600 0x7fa9281d9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa927fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa928070820 0x7fa928137e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa9277fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa928070c00 0x7fa9281340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92cbe2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa92813d600 0x7fa9281d9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56154/0 (socket says 172.21.15.132:56154) 2024-03-13T08:11:15.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92cbe2640 1 -- 172.21.15.132:0/2841575700 learned_addr learned my addr 172.21.15.132:0/2841575700 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:15.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92cbe2640 10 monclient(hunting): get_auth_request con 0x7fa92813d600 auth_method 0 2024-03-13T08:11:15.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92cbe2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:15.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92cbe2640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa92cbe2640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa927fff640 10 monclient(hunting): get_auth_request con 0x7fa928070820 auth_method 0 2024-03-13T08:11:15.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa927fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:15.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa927fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa927fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa9277fe640 10 monclient(hunting): get_auth_request con 0x7fa928070c00 auth_method 0 2024-03-13T08:11:15.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa9277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:15.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa9277fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:15.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.896+0000 7fa9277fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:15.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.897+0000 7fa92cbe2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:15.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.897+0000 7fa92cbe2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:15.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.897+0000 7fa92cbe2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:15.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.897+0000 7fa9277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:15.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.897+0000 7fa9277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:15.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.897+0000 7fa9277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:15.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.897+0000 7fa927fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:15.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.897+0000 7fa927fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:15.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.897+0000 7fa927fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:15.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.899+0000 7fa9277fe640 10 monclient(hunting): handle_auth_done global_id 19708 payload 824 2024-03-13T08:11:15.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.899+0000 7fa9277fe640 1 -- 172.21.15.132:0/2841575700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa928070820 msgr2=0x7fa928137e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.899+0000 7fa9277fe640 1 --2- 172.21.15.132:0/2841575700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa928070820 0x7fa928137e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.899+0000 7fa9277fe640 1 -- 172.21.15.132:0/2841575700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa92813d600 msgr2=0x7fa9281d9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:15.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.899+0000 7fa9277fe640 1 --2- 172.21.15.132:0/2841575700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa92813d600 0x7fa9281d9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:15.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.899+0000 7fa9277fe640 10 monclient: _finish_hunting 0 2024-03-13T08:11:15.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.899+0000 7fa9277fe640 1 monclient: found mon.noname-b 2024-03-13T08:11:15.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.899+0000 7fa9277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:15.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.899+0000 7fa9277fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T08:11:15.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.899+0000 7fa9277fe640 1 -- 172.21.15.132:0/2841575700 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92813e960 con 0x7fa928070c00 2024-03-13T08:11:15.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.902+0000 7fa9277fe640 1 --2- 172.21.15.132:0/2841575700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa928070c00 0x7fa9281340a0 secure :-1 s=READY pgs=4737 cs=0 l=1 rev1=1 crypto rx=0x7fa91c033000 tx=0x7fa91c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:15.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.902+0000 7fa924ff9640 10 client.?.objecter ms_handle_connect 0x7fa928070c00 2024-03-13T08:11:15.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.902+0000 7fa924ff9640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:15.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa924ff9640 1 -- 172.21.15.132:0/2841575700 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c032630 con 0x7fa928070c00 2024-03-13T08:11:15.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa924ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:15.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa924ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:15.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa924ff9640 10 monclient: dump: 2024-03-13T08:11:15.909 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:15.909 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:15.909 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.909 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.909 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:15.909 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.910 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:15.910 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:15.910 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:15.910 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa924ff9640 10 monclient: _finish_auth 0 2024-03-13T08:11:15.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa924ff9640 10 monclient: _check_auth_tickets 2024-03-13T08:11:15.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa924ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:15.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa924ff9640 1 -- 172.21.15.132:0/2841575700 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91c0327f0 con 0x7fa928070c00 2024-03-13T08:11:15.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa92e645640 5 monclient: authenticate success, global_id 19708 2024-03-13T08:11:15.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa924ff9640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:15.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa924ff9640 1 -- 172.21.15.132:0/2841575700 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c032be0 con 0x7fa928070c00 2024-03-13T08:11:15.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa924ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:15.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa924ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:15.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa924ff9640 10 monclient: dump: 2024-03-13T08:11:15.911 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:15.911 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:15.911 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.911 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:15.911 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:15.911 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:15.911 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:15.911 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:15.911 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:15.911 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:15.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa92e645640 10 monclient: _renew_subs 2024-03-13T08:11:15.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa92e645640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:15.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa9267fc640 4 set_mon_vals no callback set 2024-03-13T08:11:15.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa92e645640 1 -- 172.21.15.132:0/2841575700 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92813ec40 con 0x7fa928070c00 2024-03-13T08:11:15.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa92e645640 10 client.19708.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:15.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa92e645640 10 monclient: _renew_subs 2024-03-13T08:11:15.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa92e645640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:15.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa92e645640 1 -- 172.21.15.132:0/2841575700 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9281d98d0 con 0x7fa928070c00 2024-03-13T08:11:15.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.906+0000 7fa92e645640 1 librados: init done 2024-03-13T08:11:15.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.907+0000 7fa92e645640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:15.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.907+0000 7fa92e645640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:15.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.907+0000 7fa92e645640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:15.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.907+0000 7fa92e645640 1 -- 172.21.15.132:0/2841575700 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa92804f000 con 0x7fa928070c00 2024-03-13T08:11:15.926 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.925+0000 7fa924ff9640 1 -- 172.21.15.132:0/2841575700 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa91c04e430 con 0x7fa928070c00 2024-03-13T08:11:15.927 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.925+0000 7fa924ff9640 1 --2- 172.21.15.132:0/2841575700 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fa91406d590 0x7fa91406fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:15.927 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.925+0000 7fa927fff640 1 --2- 172.21.15.132:0/2841575700 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fa91406d590 0x7fa91406fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:15.927 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.925+0000 7fa927fff640 10 monclient: get_auth_request con 0x7fa91406d590 auth_method 0 2024-03-13T08:11:15.927 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.926+0000 7fa924ff9640 10 client.19708.objecter ms_handle_connect 0x7fa91406d590 2024-03-13T08:11:15.927 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.926+0000 7fa927fff640 1 --2- 172.21.15.132:0/2841575700 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fa91406d590 0x7fa91406fa50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa918009b40 tx=0x7fa918009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:15.927 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.927+0000 7fa924ff9640 1 -- 172.21.15.132:0/2841575700 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7fa91c02f030 con 0x7fa928070c00 2024-03-13T08:11:15.928 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.927+0000 7fa924ff9640 10 client.19708.objecter ms_dispatch 0x7fa928000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:15.928 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.927+0000 7fa924ff9640 3 client.19708.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:15.928 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.927+0000 7fa924ff9640 3 client.19708.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:15.928 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:15.927+0000 7fa924ff9640 20 client.19708.objecter dump_active .. 0 homeless 2024-03-13T08:11:16.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.173+0000 7fa924ff9640 1 -- 172.21.15.132:0/2841575700 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fa91c040070 con 0x7fa928070c00 2024-03-13T08:11:16.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.173+0000 7fa924ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:16.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.173+0000 7fa924ff9640 10 monclient: _finish_command 1 = system:0 2024-03-13T08:11:16.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.361+0000 7fa92e645640 1 -- 172.21.15.132:0/2841575700 --> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9281345e0 con 0x7fa91406d590 2024-03-13T08:11:16.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.362+0000 7fa924ff9640 1 -- 172.21.15.132:0/2841575700 <== mgr.4104 v2:172.21.15.132:6800/3847550149 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa9281345e0 con 0x7fa91406d590 2024-03-13T08:11:16.363 INFO:teuthology.orchestra.run.smithi132.stderr:instructed osd(s) 4 to deep-scrub 2024-03-13T08:11:16.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 10 librados: watch_flush enter 2024-03-13T08:11:16.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 10 librados: watch_flush exit 2024-03-13T08:11:16.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 20 client.19708.objecter shutdown clearing up homeless session... 2024-03-13T08:11:16.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 10 client.19708.objecter successfully canceled tick 2024-03-13T08:11:16.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 1 -- 172.21.15.132:0/2841575700 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fa91406d590 msgr2=0x7fa91406fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:16.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 1 --2- 172.21.15.132:0/2841575700 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fa91406d590 0x7fa91406fa50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa918009b40 tx=0x7fa918009190 comp rx=0 tx=0).stop 2024-03-13T08:11:16.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 10 monclient: shutdown 2024-03-13T08:11:16.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:16.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 1 -- 172.21.15.132:0/2841575700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa928070c00 msgr2=0x7fa9281340a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:16.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 1 --2- 172.21.15.132:0/2841575700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa928070c00 0x7fa9281340a0 secure :-1 s=READY pgs=4737 cs=0 l=1 rev1=1 crypto rx=0x7fa91c033000 tx=0x7fa91c0046c0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 1 -- 172.21.15.132:0/2841575700 shutdown_connections 2024-03-13T08:11:16.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 1 --2- 172.21.15.132:0/2841575700 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fa91406d590 0x7fa91406fa50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 1 --2- 172.21.15.132:0/2841575700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa92813d600 0x7fa9281d9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 1 --2- 172.21.15.132:0/2841575700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa928070c00 0x7fa9281340a0 unknown :-1 s=CLOSED pgs=4737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 1 --2- 172.21.15.132:0/2841575700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa928070820 0x7fa928137e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 1 -- 172.21.15.132:0/2841575700 >> 172.21.15.132:0/2841575700 conn(0x7fa9280b3e20 msgr2=0x7fa92813b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:16.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 1 -- 172.21.15.132:0/2841575700 shutdown_connections 2024-03-13T08:11:16.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 1 -- 172.21.15.132:0/2841575700 wait complete. 2024-03-13T08:11:16.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.364+0000 7fa92e645640 1 librados: shutdown 2024-03-13T08:11:16.376 INFO:tasks.ceph:Scrubbing osd.5 2024-03-13T08:11:16.376 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-13T08:11:16.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.482+0000 7fa05d532640 10 monclient: get_monmap_and_config 2024-03-13T08:11:16.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.482+0000 7fa05d532640 10 monclient: build_initial_monmap 2024-03-13T08:11:16.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.482+0000 7fa05d532640 10 monclient: monmap: 2024-03-13T08:11:16.482 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:16.483 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:16.483 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:16.482967+0000 2024-03-13T08:11:16.483 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:16.482967+0000 2024-03-13T08:11:16.483 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:16.483 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:16.483 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:16.483 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:16.483 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:16.483 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:16.483 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:16.483 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:16.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.482+0000 7fa05d532640 1 Processor -- start 2024-03-13T08:11:16.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.482+0000 7fa05d532640 1 -- start start 2024-03-13T08:11:16.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.482+0000 7fa05d532640 10 monclient: init 2024-03-13T08:11:16.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.483+0000 7fa05d532640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:16.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.483+0000 7fa05d532640 10 monclient: _add_conns ranks=[3,2,0,4,1] 2024-03-13T08:11:16.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.483+0000 7fa05d532640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa0580b8760 0x7fa0581426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:16.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.483+0000 7fa05d532640 10 monclient(hunting): picked mon.noname-a con 0x7fa0580b8760 addr v2:172.21.15.132:3300/0 2024-03-13T08:11:16.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.483+0000 7fa05d532640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580b8b40 0x7fa0580b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:16.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.483+0000 7fa05d532640 10 monclient(hunting): picked mon.noname-b con 0x7fa0580b8b40 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:16.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.483+0000 7fa05d532640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa0580b24e0 addr v1:172.21.15.132:6789/0 2024-03-13T08:11:16.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.483+0000 7fa05d532640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0580b1b00 con 0x7fa0580b24e0 2024-03-13T08:11:16.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.483+0000 7fa05d532640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:16.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.483+0000 7fa05d532640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0580b1400 con 0x7fa0580b8760 2024-03-13T08:11:16.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.483+0000 7fa05d532640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:16.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.483+0000 7fa05d532640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580b16e0 con 0x7fa0580b8b40 2024-03-13T08:11:16.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.483+0000 7fa05d532640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:16.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.483+0000 7fa05d532640 10 monclient(hunting): authenticate will time out at 3503.891573s 2024-03-13T08:11:16.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.484+0000 7fa0567fc640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580b8b40 0x7fa0580b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:16.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.484+0000 7fa056ffd640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa0580b8760 0x7fa0581426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:16.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.484+0000 7fa0577fe640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fa0580b24e0 0x7fa0580aea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:50424/0 (socket says 172.21.15.132:50424) 2024-03-13T08:11:16.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.484+0000 7fa0577fe640 1 -- 172.21.15.132:0/3349257153 learned_addr learned my addr 172.21.15.132:0/3349257153 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T08:11:16.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.484+0000 7fa0567fc640 1 --2- 172.21.15.132:0/3349257153 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580b8b40 0x7fa0580b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:59830/0 (socket says 172.21.15.132:59830) 2024-03-13T08:11:16.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.484+0000 7fa0567fc640 10 monclient(hunting): get_auth_request con 0x7fa0580b8b40 auth_method 0 2024-03-13T08:11:16.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.484+0000 7fa0567fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:16.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.484+0000 7fa0567fc640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:16.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.484+0000 7fa0567fc640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:16.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.484+0000 7fa056ffd640 10 monclient(hunting): get_auth_request con 0x7fa0580b8760 auth_method 0 2024-03-13T08:11:16.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.484+0000 7fa056ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:16.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.484+0000 7fa056ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:16.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.484+0000 7fa056ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:16.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.484+0000 7fa0567fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:16.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.485+0000 7fa0567fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:16.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.485+0000 7fa0567fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:16.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.485+0000 7fa056ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:16.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.485+0000 7fa056ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:16.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.485+0000 7fa056ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:16.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.486+0000 7fa0567fc640 10 monclient(hunting): handle_auth_done global_id 19013 payload 274 2024-03-13T08:11:16.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.486+0000 7fa0567fc640 1 -- 172.21.15.132:0/3349257153 >> v2:172.21.15.132:3300/0 conn(0x7fa0580b8760 msgr2=0x7fa0581426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:16.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.486+0000 7fa0567fc640 1 --2- 172.21.15.132:0/3349257153 >> v2:172.21.15.132:3300/0 conn(0x7fa0580b8760 0x7fa0581426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.486+0000 7fa0567fc640 1 -- 172.21.15.132:0/3349257153 >> v1:172.21.15.132:6789/0 conn(0x7fa0580b24e0 legacy=0x7fa0580aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:16.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.486+0000 7fa0567fc640 10 monclient: _finish_hunting 0 2024-03-13T08:11:16.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.486+0000 7fa0567fc640 1 monclient: found mon.noname-b 2024-03-13T08:11:16.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.486+0000 7fa0567fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:16.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.486+0000 7fa0567fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T08:11:16.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.486+0000 7fa0567fc640 1 -- 172.21.15.132:0/3349257153 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0580b4840 con 0x7fa0580b8b40 2024-03-13T08:11:16.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.488+0000 7fa0567fc640 1 --2- 172.21.15.132:0/3349257153 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580b8b40 0x7fa0580b1fa0 secure :-1 s=READY pgs=3542 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0097a0 tx=0x7fa04c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e6a64857c332901 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:16.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa055ffb640 1 -- 172.21.15.132:0/3349257153 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c0315a0 con 0x7fa0580b8b40 2024-03-13T08:11:16.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa055ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:16.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa055ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:16.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa055ffb640 10 monclient: dump: 2024-03-13T08:11:16.508 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:16.508 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:16.508 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:16.508 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:16.509 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:16.509 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:16.509 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:16.509 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:16.509 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:16.511 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:16.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa055ffb640 10 monclient: _finish_auth 0 2024-03-13T08:11:16.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa055ffb640 10 monclient: _check_auth_tickets 2024-03-13T08:11:16.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa055ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:16.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa055ffb640 1 -- 172.21.15.132:0/3349257153 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c037070 con 0x7fa0580b8b40 2024-03-13T08:11:16.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa05d532640 5 monclient: authenticate success, global_id 19013 2024-03-13T08:11:16.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa05d532640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T08:11:16.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa055ffb640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:16.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa05d532640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:16.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa05d532640 4 set_mon_vals no callback set 2024-03-13T08:11:16.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa055ffb640 1 -- 172.21.15.132:0/3349257153 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c031960 con 0x7fa0580b8b40 2024-03-13T08:11:16.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa055ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:16.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa055ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T08:11:16.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.492+0000 7fa055ffb640 10 monclient: dump: 2024-03-13T08:11:16.512 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:16.512 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:16.512 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:16.512 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:16.513 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:16.513 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:16.513 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:16.513 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:16.513 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:16.513 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:16.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.493+0000 7fa05d532640 10 monclient: shutdown 2024-03-13T08:11:16.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.493+0000 7fa05d532640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:16.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.493+0000 7fa05d532640 1 -- 172.21.15.132:0/3349257153 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580b8b40 msgr2=0x7fa0580b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:16.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.493+0000 7fa05d532640 1 --2- 172.21.15.132:0/3349257153 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580b8b40 0x7fa0580b1fa0 secure :-1 s=READY pgs=3542 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0097a0 tx=0x7fa04c02b840 comp rx=0 tx=0).stop 2024-03-13T08:11:16.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.493+0000 7fa05d532640 1 -- 172.21.15.132:0/3349257153 shutdown_connections 2024-03-13T08:11:16.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.493+0000 7fa05d532640 1 --2- 172.21.15.132:0/3349257153 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580b8b40 0x7fa0580b1fa0 unknown :-1 s=CLOSED pgs=3542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.493+0000 7fa05d532640 1 --2- 172.21.15.132:0/3349257153 >> v2:172.21.15.132:3300/0 conn(0x7fa0580b8760 0x7fa0581426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.493+0000 7fa05d532640 1 -- 172.21.15.132:0/3349257153 >> 172.21.15.132:0/3349257153 conn(0x7fa05806d4f0 msgr2=0x7fa0580b8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:16.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.493+0000 7fa05d532640 1 -- 172.21.15.132:0/3349257153 shutdown_connections 2024-03-13T08:11:16.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.493+0000 7fa05d532640 1 -- 172.21.15.132:0/3349257153 wait complete. 2024-03-13T08:11:16.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.494+0000 7fa05d532640 10 monclient: build_initial_monmap 2024-03-13T08:11:16.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.494+0000 7fa05d532640 10 monclient: monmap: 2024-03-13T08:11:16.514 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:16.514 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:16.514 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:16.514 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:16.514 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:16.514 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:16.514 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:16.514 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:16.514 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:16.514 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:16.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.494+0000 7fa05d532640 1 librados: starting msgr at 2024-03-13T08:11:16.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.494+0000 7fa05d532640 1 librados: starting objecter 2024-03-13T08:11:16.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 1 Processor -- start 2024-03-13T08:11:16.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 1 -- start start 2024-03-13T08:11:16.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 1 librados: setting wanted keys 2024-03-13T08:11:16.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 1 librados: calling monclient init 2024-03-13T08:11:16.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 10 monclient: init 2024-03-13T08:11:16.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:16.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 10 monclient: _add_conns ranks=[0,1,2] 2024-03-13T08:11:16.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0580b8760 0x7fa05813e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:16.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 10 monclient(hunting): picked mon.noname-a con 0x7fa0580b8760 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:16.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0580ad770 0x7fa0580aac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:16.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 10 monclient(hunting): picked mon.noname-b con 0x7fa0580ad770 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:16.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580ab920 0x7fa0580abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:16.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 10 monclient(hunting): picked mon.noname-c con 0x7fa0580ab920 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:16.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:16.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580b00c0 con 0x7fa0580b8760 2024-03-13T08:11:16.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:16.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580aff40 con 0x7fa0580ad770 2024-03-13T08:11:16.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:16.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580b0240 con 0x7fa0580ab920 2024-03-13T08:11:16.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:16.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.495+0000 7fa05d532640 10 monclient(hunting): authenticate will time out at 3503.903864s 2024-03-13T08:11:16.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa0577fe640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580ab920 0x7fa0580abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:16.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa056ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0580b8760 0x7fa05813e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:16.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa056ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0580b8760 0x7fa05813e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56172/0 (socket says 172.21.15.132:56172) 2024-03-13T08:11:16.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa056ffd640 1 -- 172.21.15.132:0/1921839284 learned_addr learned my addr 172.21.15.132:0/1921839284 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:16.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa056ffd640 10 monclient(hunting): get_auth_request con 0x7fa0580b8760 auth_method 0 2024-03-13T08:11:16.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa056ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:16.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa056ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:16.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa056ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:16.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa0567fc640 1 --2- 172.21.15.132:0/1921839284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0580ad770 0x7fa0580aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:16.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa0577fe640 10 monclient(hunting): get_auth_request con 0x7fa0580ab920 auth_method 0 2024-03-13T08:11:16.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa0577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:16.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa0577fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:16.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa0577fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:16.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa0567fc640 10 monclient(hunting): get_auth_request con 0x7fa0580ad770 auth_method 0 2024-03-13T08:11:16.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa0567fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:16.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa0567fc640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:16.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.496+0000 7fa0567fc640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:16.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.497+0000 7fa056ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:16.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.497+0000 7fa056ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:16.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.497+0000 7fa056ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:16.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.497+0000 7fa0577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:16.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.497+0000 7fa0577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:16.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.497+0000 7fa0577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:16.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.497+0000 7fa0567fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:16.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.497+0000 7fa0567fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:16.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.497+0000 7fa0567fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:16.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.499+0000 7fa0567fc640 10 monclient(hunting): handle_auth_done global_id 19711 payload 824 2024-03-13T08:11:16.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.499+0000 7fa0567fc640 1 -- 172.21.15.132:0/1921839284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580ab920 msgr2=0x7fa0580abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:16.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.499+0000 7fa0567fc640 1 --2- 172.21.15.132:0/1921839284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0580ab920 0x7fa0580abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.499+0000 7fa0567fc640 1 -- 172.21.15.132:0/1921839284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0580b8760 msgr2=0x7fa05813e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:16.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.499+0000 7fa0567fc640 1 --2- 172.21.15.132:0/1921839284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0580b8760 0x7fa05813e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.500+0000 7fa0567fc640 10 monclient: _finish_hunting 0 2024-03-13T08:11:16.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.500+0000 7fa0567fc640 1 monclient: found mon.noname-b 2024-03-13T08:11:16.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.500+0000 7fa0567fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:16.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.500+0000 7fa0567fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T08:11:16.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.500+0000 7fa0567fc640 1 -- 172.21.15.132:0/1921839284 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0580ab3e0 con 0x7fa0580ad770 2024-03-13T08:11:16.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.500+0000 7fa056ffd640 1 --2- 172.21.15.132:0/1921839284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0580b8760 0x7fa05813e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-13T08:11:16.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.502+0000 7fa0567fc640 1 --2- 172.21.15.132:0/1921839284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0580ad770 0x7fa0580aac80 secure :-1 s=READY pgs=4738 cs=0 l=1 rev1=1 crypto rx=0x7fa04c033000 tx=0x7fa04c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:16.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.502+0000 7fa03f7fe640 10 client.?.objecter ms_handle_connect 0x7fa0580ad770 2024-03-13T08:11:16.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.502+0000 7fa03f7fe640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:16.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa03f7fe640 1 -- 172.21.15.132:0/1921839284 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c032630 con 0x7fa0580ad770 2024-03-13T08:11:16.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa03f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:16.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa03f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:16.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa03f7fe640 10 monclient: dump: 2024-03-13T08:11:16.520 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:16.521 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:16.521 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:16.521 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:16.521 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:16.521 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:16.521 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:16.521 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:16.521 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:16.521 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:16.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa03f7fe640 10 monclient: _finish_auth 0 2024-03-13T08:11:16.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa03f7fe640 10 monclient: _check_auth_tickets 2024-03-13T08:11:16.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa03f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:16.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa03f7fe640 1 -- 172.21.15.132:0/1921839284 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c0327f0 con 0x7fa0580ad770 2024-03-13T08:11:16.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa03f7fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:16.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa03f7fe640 1 -- 172.21.15.132:0/1921839284 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c032be0 con 0x7fa0580ad770 2024-03-13T08:11:16.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa05d532640 5 monclient: authenticate success, global_id 19711 2024-03-13T08:11:16.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa03f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:16.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa03f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:16.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa03f7fe640 10 monclient: dump: 2024-03-13T08:11:16.522 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:16.522 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:16.522 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:16.522 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:16.522 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:16.522 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:16.523 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:16.523 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:16.523 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:16.523 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:16.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa0557fa640 4 set_mon_vals no callback set 2024-03-13T08:11:16.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa05d532640 10 monclient: _renew_subs 2024-03-13T08:11:16.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa05d532640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:16.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.505+0000 7fa05d532640 1 -- 172.21.15.132:0/1921839284 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0580ad2e0 con 0x7fa0580ad770 2024-03-13T08:11:16.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.506+0000 7fa05d532640 10 client.19711.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:16.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.506+0000 7fa05d532640 10 monclient: _renew_subs 2024-03-13T08:11:16.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.506+0000 7fa05d532640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:16.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.506+0000 7fa05d532640 1 -- 172.21.15.132:0/1921839284 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa058137420 con 0x7fa0580ad770 2024-03-13T08:11:16.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.506+0000 7fa05d532640 1 librados: init done 2024-03-13T08:11:16.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.506+0000 7fa05d532640 10 client.19711.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:16.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.506+0000 7fa05d532640 15 client.19711.objecter _session_command_op_assign -1 1 2024-03-13T08:11:16.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.506+0000 7fa05d532640 20 client.19711.objecter _get_session osd=-1 returning homeless 2024-03-13T08:11:16.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.506+0000 7fa05d532640 10 client.19711.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:16.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.506+0000 7fa05d532640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:16.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.506+0000 7fa05d532640 1 -- 172.21.15.132:0/1921839284 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa05804f000 con 0x7fa0580ad770 2024-03-13T08:11:16.526 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.525+0000 7fa03f7fe640 1 -- 172.21.15.132:0/1921839284 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fa04c04d3f0 con 0x7fa0580ad770 2024-03-13T08:11:16.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.525+0000 7fa03f7fe640 1 --2- 172.21.15.132:0/1921839284 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fa04406d5b0 0x7fa04406fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:16.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.526+0000 7fa056ffd640 1 --2- 172.21.15.132:0/1921839284 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fa04406d5b0 0x7fa04406fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:16.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.526+0000 7fa056ffd640 10 monclient: get_auth_request con 0x7fa04406d5b0 auth_method 0 2024-03-13T08:11:16.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.526+0000 7fa056ffd640 1 --2- 172.21.15.132:0/1921839284 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fa04406d5b0 0x7fa04406fa70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa048009a40 tx=0x7fa04800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:16.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.526+0000 7fa03f7fe640 10 client.19711.objecter ms_handle_connect 0x7fa04406d5b0 2024-03-13T08:11:16.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 1 -- 172.21.15.132:0/1921839284 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7fa04c02f030 con 0x7fa0580ad770 2024-03-13T08:11:16.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 10 client.19711.objecter ms_dispatch 0x7fa058000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:16.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 3 client.19711.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:16.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 3 client.19711.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:16.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 10 client.19711.objecter checking command 1 2024-03-13T08:11:16.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 1 --2- 172.21.15.132:0/1921839284 >> [v2:172.21.15.195:6816/1953907706,v1:172.21.15.195:6817/1953907706] conn(0x7fa044075220 0x7fa0440776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:16.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 20 client.19711.objecter _get_session s=0x7fa044074b50 osd=5 3 2024-03-13T08:11:16.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 20 client.19711.objecter put_session s=0x7fa044074b50 osd=5 3 2024-03-13T08:11:16.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 15 client.19711.objecter _session_command_op_remove -1 1 2024-03-13T08:11:16.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 20 client.19711.objecter _get_session s=0x7fa044074b50 osd=5 3 2024-03-13T08:11:16.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 20 client.19711.objecter get_session s=0x7fa044074b50 osd=5 3 2024-03-13T08:11:16.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 15 client.19711.objecter _session_command_op_assign 5 1 2024-03-13T08:11:16.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 20 client.19711.objecter put_session s=0x7fa044074b50 osd=5 4 2024-03-13T08:11:16.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 10 client.19711.objecter _send_command 1 2024-03-13T08:11:16.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 1 -- 172.21.15.132:0/1921839284 --> [v2:172.21.15.195:6816/1953907706,v1:172.21.15.195:6817/1953907706] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa04c02de50 con 0x7fa044075220 2024-03-13T08:11:16.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 20 client.19711.objecter dump_active .. 0 homeless 2024-03-13T08:11:16.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 1 -- 172.21.15.132:0/1921839284 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=1293) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa04c077060 con 0x7fa0580ad770 2024-03-13T08:11:16.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.528+0000 7fa03f7fe640 10 monclient: handle_get_version_reply finishing 1 version 1293 2024-03-13T08:11:16.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.529+0000 7fa0577fe640 1 --2- 172.21.15.132:0/1921839284 >> [v2:172.21.15.195:6816/1953907706,v1:172.21.15.195:6817/1953907706] conn(0x7fa044075220 0x7fa0440776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:16.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.529+0000 7fa0577fe640 10 monclient: get_auth_request con 0x7fa044075220 auth_method 0 2024-03-13T08:11:16.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.532+0000 7fa0577fe640 1 --2- 172.21.15.132:0/1921839284 >> [v2:172.21.15.195:6816/1953907706,v1:172.21.15.195:6817/1953907706] conn(0x7fa044075220 0x7fa0440776c0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:16.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.532+0000 7fa03f7fe640 10 client.19711.objecter ms_handle_connect 0x7fa044075220 2024-03-13T08:11:16.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.556+0000 7fa03f7fe640 1 -- 172.21.15.132:0/1921839284 <== osd.5 v2:172.21.15.195:6816/1953907706 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa04c02de50 con 0x7fa044075220 2024-03-13T08:11:16.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.556+0000 7fa03f7fe640 10 client.19711.objecter ms_dispatch 0x7fa058000b90 command_reply(tid 1: 0 ) v1 2024-03-13T08:11:16.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.556+0000 7fa03f7fe640 10 client.19711.objecter _finish_command 1 = system:0 2024-03-13T08:11:16.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.556+0000 7fa03f7fe640 20 client.19711.objecter put_session s=0x7fa044074b50 osd=5 4 2024-03-13T08:11:16.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.556+0000 7fa03f7fe640 15 client.19711.objecter _session_command_op_remove 5 1 2024-03-13T08:11:16.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.570+0000 7fa05d532640 10 client.19711.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-03-13T08:11:16.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.570+0000 7fa05d532640 15 client.19711.objecter _session_command_op_assign -1 2 2024-03-13T08:11:16.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.570+0000 7fa05d532640 20 client.19711.objecter _get_session s=0x7fa044074b50 osd=5 3 2024-03-13T08:11:16.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.570+0000 7fa05d532640 20 client.19711.objecter put_session s=0x7fa044074b50 osd=5 3 2024-03-13T08:11:16.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.570+0000 7fa05d532640 20 client.19711.objecter _get_session s=0x7fa044074b50 osd=5 3 2024-03-13T08:11:16.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.570+0000 7fa05d532640 15 client.19711.objecter _session_command_op_remove -1 2 2024-03-13T08:11:16.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.570+0000 7fa05d532640 20 client.19711.objecter get_session s=0x7fa044074b50 osd=5 3 2024-03-13T08:11:16.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.570+0000 7fa05d532640 15 client.19711.objecter _session_command_op_assign 5 2 2024-03-13T08:11:16.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.570+0000 7fa05d532640 20 client.19711.objecter put_session s=0x7fa044074b50 osd=5 4 2024-03-13T08:11:16.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.570+0000 7fa05d532640 10 client.19711.objecter _send_command 2 2024-03-13T08:11:16.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.570+0000 7fa05d532640 1 -- 172.21.15.132:0/1921839284 --> [v2:172.21.15.195:6816/1953907706,v1:172.21.15.195:6817/1953907706] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa0580ab620 con 0x7fa044075220 2024-03-13T08:11:16.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa03f7fe640 1 -- 172.21.15.132:0/1921839284 <== osd.5 v2:172.21.15.195:6816/1953907706 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa050004b00 con 0x7fa044075220 2024-03-13T08:11:16.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa03f7fe640 10 client.19711.objecter ms_dispatch 0x7fa058000b90 command_reply(tid 2: 0 ) v1 2024-03-13T08:11:16.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa03f7fe640 10 client.19711.objecter _finish_command 2 = system:0 2024-03-13T08:11:16.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa03f7fe640 20 client.19711.objecter put_session s=0x7fa044074b50 osd=5 4 2024-03-13T08:11:16.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa03f7fe640 15 client.19711.objecter _session_command_op_remove 5 2 2024-03-13T08:11:16.585 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-13T08:11:16.585 INFO:teuthology.orchestra.run.smithi132.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-13T08:11:16.585 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-13T08:11:16.586 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 10 librados: watch_flush enter 2024-03-13T08:11:16.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 10 librados: watch_flush exit 2024-03-13T08:11:16.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 10 client.19711.objecter close_session for osd.5 2024-03-13T08:11:16.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 1 -- 172.21.15.132:0/1921839284 >> [v2:172.21.15.195:6816/1953907706,v1:172.21.15.195:6817/1953907706] conn(0x7fa044075220 msgr2=0x7fa0440776c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:16.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 1 --2- 172.21.15.132:0/1921839284 >> [v2:172.21.15.195:6816/1953907706,v1:172.21.15.195:6817/1953907706] conn(0x7fa044075220 0x7fa0440776c0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 20 client.19711.objecter put_session s=0x7fa044074b50 osd=5 1 2024-03-13T08:11:16.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 20 client.19711.objecter shutdown clearing up homeless session... 2024-03-13T08:11:16.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 10 client.19711.objecter successfully canceled tick 2024-03-13T08:11:16.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 1 -- 172.21.15.132:0/1921839284 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fa04406d5b0 msgr2=0x7fa04406fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:16.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 1 --2- 172.21.15.132:0/1921839284 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fa04406d5b0 0x7fa04406fa70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa048009a40 tx=0x7fa04800b040 comp rx=0 tx=0).stop 2024-03-13T08:11:16.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 10 monclient: shutdown 2024-03-13T08:11:16.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:16.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 1 -- 172.21.15.132:0/1921839284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0580ad770 msgr2=0x7fa0580aac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:16.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 1 --2- 172.21.15.132:0/1921839284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0580ad770 0x7fa0580aac80 secure :-1 s=READY pgs=4738 cs=0 l=1 rev1=1 crypto rx=0x7fa04c033000 tx=0x7fa04c0046c0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa0577fe640 1 -- 172.21.15.132:0/1921839284 reap_dead start 2024-03-13T08:11:16.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 1 -- 172.21.15.132:0/1921839284 shutdown_connections 2024-03-13T08:11:16.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 1 -- 172.21.15.132:0/1921839284 >> 172.21.15.132:0/1921839284 conn(0x7fa05806d4f0 msgr2=0x7fa05813ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:16.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 1 -- 172.21.15.132:0/1921839284 shutdown_connections 2024-03-13T08:11:16.589 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 1 -- 172.21.15.132:0/1921839284 wait complete. 2024-03-13T08:11:16.589 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.585+0000 7fa05d532640 1 librados: shutdown 2024-03-13T08:11:16.597 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-13T08:11:16.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.702+0000 7f58cb5ec640 10 monclient: get_monmap_and_config 2024-03-13T08:11:16.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.702+0000 7f58cb5ec640 10 monclient: build_initial_monmap 2024-03-13T08:11:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.702+0000 7f58cb5ec640 10 monclient: monmap: 2024-03-13T08:11:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:16.703261+0000 2024-03-13T08:11:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:16.703261+0000 2024-03-13T08:11:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:16.703 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:16.703 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:16.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.703+0000 7f58cb5ec640 1 Processor -- start 2024-03-13T08:11:16.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.703+0000 7f58cb5ec640 1 -- start start 2024-03-13T08:11:16.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.703+0000 7f58cb5ec640 10 monclient: init 2024-03-13T08:11:16.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.704+0000 7f58cb5ec640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:16.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.704+0000 7f58cb5ec640 10 monclient: _add_conns ranks=[1,4,2,3,0] 2024-03-13T08:11:16.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.704+0000 7f58cb5ec640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f58c406fb50 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:16.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.704+0000 7f58cb5ec640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f58c406ff30 0x7f58c406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:16.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.704+0000 7f58cb5ec640 10 monclient(hunting): picked mon.noname-c con 0x7f58c406ff30 addr v2:172.21.15.195:3300/0 2024-03-13T08:11:16.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.704+0000 7f58cb5ec640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c4071cf0 0x7f58c4072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:16.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.704+0000 7f58cb5ec640 10 monclient(hunting): picked mon.noname-b con 0x7f58c4071cf0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:16.706 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.704+0000 7f58cb5ec640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c4071830 con 0x7f58c406fb50 2024-03-13T08:11:16.706 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.704+0000 7f58cb5ec640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:16.706 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.704+0000 7f58cb5ec640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c4071130 con 0x7f58c406ff30 2024-03-13T08:11:16.706 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.704+0000 7f58cb5ec640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:16.706 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.704+0000 7f58cb5ec640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4071410 con 0x7f58c4071cf0 2024-03-13T08:11:16.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.704+0000 7f58cb5ec640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:16.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.704+0000 7f58cb5ec640 10 monclient(hunting): authenticate will time out at 3504.112257s 2024-03-13T08:11:16.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.704+0000 7f58c9b89640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c4071cf0 0x7f58c4072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:16.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.705+0000 7f58c9b89640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c4071cf0 0x7f58c4072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:59838/0 (socket says 172.21.15.132:59838) 2024-03-13T08:11:16.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.705+0000 7f58c8b87640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f58c406ff30 0x7f58c406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:16.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.705+0000 7f58c9b89640 1 -- 172.21.15.132:0/3893471963 learned_addr learned my addr 172.21.15.132:0/3893471963 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:16.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.705+0000 7f58c9b89640 10 monclient(hunting): get_auth_request con 0x7f58c4071cf0 auth_method 0 2024-03-13T08:11:16.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.705+0000 7f58c9b89640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:16.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.705+0000 7f58c9b89640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:16.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.705+0000 7f58c9b89640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:16.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.705+0000 7f58c8b87640 10 monclient(hunting): get_auth_request con 0x7f58c406ff30 auth_method 0 2024-03-13T08:11:16.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.705+0000 7f58c8b87640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:16.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.705+0000 7f58c8b87640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:16.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.705+0000 7f58c8b87640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:16.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.705+0000 7f58c9b89640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:16.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.705+0000 7f58c9b89640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:16.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.705+0000 7f58c9b89640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:16.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.706+0000 7f58c8b87640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:16.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.706+0000 7f58c8b87640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:16.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.706+0000 7f58c8b87640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:16.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.707+0000 7f58c9b89640 10 monclient(hunting): handle_auth_done global_id 19019 payload 274 2024-03-13T08:11:16.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.707+0000 7f58c9b89640 1 -- 172.21.15.132:0/3893471963 >> v2:172.21.15.195:3300/0 conn(0x7f58c406ff30 msgr2=0x7f58c406f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:16.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.707+0000 7f58c9b89640 1 --2- 172.21.15.132:0/3893471963 >> v2:172.21.15.195:3300/0 conn(0x7f58c406ff30 0x7f58c406f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.707+0000 7f58c9b89640 1 -- 172.21.15.132:0/3893471963 >> v1:172.21.15.195:6789/0 conn(0x7f58c406fb50 legacy=0x7f58c413f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:16.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.707+0000 7f58c9b89640 10 monclient: _finish_hunting 0 2024-03-13T08:11:16.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.707+0000 7f58c9b89640 1 monclient: found mon.noname-b 2024-03-13T08:11:16.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.707+0000 7f58c9b89640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:16.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.707+0000 7f58c9b89640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T08:11:16.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.707+0000 7f58c9b89640 1 -- 172.21.15.132:0/3893471963 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c413e9e0 con 0x7f58c4071cf0 2024-03-13T08:11:16.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.709+0000 7f58c9b89640 1 --2- 172.21.15.132:0/3893471963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c4071cf0 0x7f58c4072160 secure :-1 s=READY pgs=3543 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009e50 tx=0x7f58bc02be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc769e9c4d770f4f server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:16.713 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.712+0000 7f58bbfff640 1 -- 172.21.15.132:0/3893471963 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc02c9e0 con 0x7f58c4071cf0 2024-03-13T08:11:16.713 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.712+0000 7f58bbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:16.713 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.712+0000 7f58bbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:16.713 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.712+0000 7f58bbfff640 10 monclient: dump: 2024-03-13T08:11:16.713 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:16.713 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:16.713 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:16.713 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:16.713 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:16.713 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:16.714 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:16.714 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:16.714 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:16.714 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:16.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.712+0000 7f58bbfff640 10 monclient: _finish_auth 0 2024-03-13T08:11:16.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.712+0000 7f58bbfff640 10 monclient: _check_auth_tickets 2024-03-13T08:11:16.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.712+0000 7f58bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:16.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.712+0000 7f58bbfff640 1 -- 172.21.15.132:0/3893471963 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58bc02cb80 con 0x7f58c4071cf0 2024-03-13T08:11:16.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.712+0000 7f58bbfff640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:16.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.712+0000 7f58bbfff640 1 -- 172.21.15.132:0/3893471963 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc0343f0 con 0x7f58c4071cf0 2024-03-13T08:11:16.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.712+0000 7f58cb5ec640 5 monclient: authenticate success, global_id 19019 2024-03-13T08:11:16.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.712+0000 7f58cb5ec640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:16.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.712+0000 7f58cb5ec640 4 set_mon_vals no callback set 2024-03-13T08:11:16.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.712+0000 7f58bbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:16.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.712+0000 7f58bbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T08:11:16.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.712+0000 7f58bbfff640 10 monclient: dump: 2024-03-13T08:11:16.715 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:16.715 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:16.715 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:16.716 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:16.716 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:16.716 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:16.716 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:16.716 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:16.716 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:16.716 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:16.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.713+0000 7f58cb5ec640 10 monclient: shutdown 2024-03-13T08:11:16.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.713+0000 7f58cb5ec640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:16.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.713+0000 7f58cb5ec640 1 -- 172.21.15.132:0/3893471963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c4071cf0 msgr2=0x7f58c4072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:16.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.713+0000 7f58cb5ec640 1 --2- 172.21.15.132:0/3893471963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c4071cf0 0x7f58c4072160 secure :-1 s=READY pgs=3543 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009e50 tx=0x7f58bc02be40 comp rx=0 tx=0).stop 2024-03-13T08:11:16.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.713+0000 7f58cb5ec640 1 -- 172.21.15.132:0/3893471963 shutdown_connections 2024-03-13T08:11:16.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.713+0000 7f58cb5ec640 1 --2- 172.21.15.132:0/3893471963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c4071cf0 0x7f58c4072160 unknown :-1 s=CLOSED pgs=3543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.713+0000 7f58cb5ec640 1 --2- 172.21.15.132:0/3893471963 >> v2:172.21.15.195:3300/0 conn(0x7f58c406ff30 0x7f58c406f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.713+0000 7f58cb5ec640 1 -- 172.21.15.132:0/3893471963 >> 172.21.15.132:0/3893471963 conn(0x7f58c40b3e20 msgr2=0x7f58c40b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:16.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.713+0000 7f58cb5ec640 1 -- 172.21.15.132:0/3893471963 shutdown_connections 2024-03-13T08:11:16.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.713+0000 7f58cb5ec640 1 -- 172.21.15.132:0/3893471963 wait complete. 2024-03-13T08:11:16.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.714+0000 7f58cb5ec640 10 monclient: build_initial_monmap 2024-03-13T08:11:16.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.714+0000 7f58cb5ec640 10 monclient: monmap: 2024-03-13T08:11:16.721 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:16.721 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:16.721 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:16.721 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:16.721 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:16.721 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:16.721 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:16.721 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:16.722 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:16.722 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:16.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.714+0000 7f58cb5ec640 1 librados: starting msgr at 2024-03-13T08:11:16.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.714+0000 7f58cb5ec640 1 librados: starting objecter 2024-03-13T08:11:16.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.714+0000 7f58cb5ec640 1 Processor -- start 2024-03-13T08:11:16.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.714+0000 7f58cb5ec640 1 -- start start 2024-03-13T08:11:16.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.714+0000 7f58cb5ec640 1 librados: setting wanted keys 2024-03-13T08:11:16.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.714+0000 7f58cb5ec640 1 librados: calling monclient init 2024-03-13T08:11:16.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.714+0000 7f58cb5ec640 10 monclient: init 2024-03-13T08:11:16.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58cb5ec640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:16.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58cb5ec640 10 monclient: _add_conns ranks=[2,0,1] 2024-03-13T08:11:16.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58cb5ec640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c406fb50 0x7f58c413a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:16.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58cb5ec640 10 monclient(hunting): picked mon.noname-c con 0x7f58c406fb50 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:16.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58cb5ec640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58c4071cf0 0x7f58c413ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:16.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58cb5ec640 10 monclient(hunting): picked mon.noname-a con 0x7f58c4071cf0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:16.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58cb5ec640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58c413a090 0x7f58c4141490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:16.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58cb5ec640 10 monclient(hunting): picked mon.noname-b con 0x7f58c413a090 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:16.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58cb5ec640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:16.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58cb5ec640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c40aea50 con 0x7f58c4071cf0 2024-03-13T08:11:16.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58cb5ec640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:16.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58cb5ec640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c40ae8d0 con 0x7f58c413a090 2024-03-13T08:11:16.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58cb5ec640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:16.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58cb5ec640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c40aebd0 con 0x7f58c406fb50 2024-03-13T08:11:16.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58cb5ec640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:16.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58cb5ec640 10 monclient(hunting): authenticate will time out at 3504.123462s 2024-03-13T08:11:16.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58c9388640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c406fb50 0x7f58c413a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:16.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58c8b87640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58c4071cf0 0x7f58c413ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:16.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.715+0000 7f58c9b89640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58c413a090 0x7f58c4141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:16.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c9b89640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58c413a090 0x7f58c4141490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.132:36070/0 (socket says 172.21.15.132:36070) 2024-03-13T08:11:16.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c9b89640 1 -- 172.21.15.132:0/1558780331 learned_addr learned my addr 172.21.15.132:0/1558780331 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:16.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c9b89640 10 monclient(hunting): get_auth_request con 0x7f58c413a090 auth_method 0 2024-03-13T08:11:16.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c9b89640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:16.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c9b89640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:16.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c9b89640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:16.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c9388640 10 monclient(hunting): get_auth_request con 0x7f58c406fb50 auth_method 0 2024-03-13T08:11:16.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c9388640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:16.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c9388640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:16.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c9388640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:16.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c8b87640 10 monclient(hunting): get_auth_request con 0x7f58c4071cf0 auth_method 0 2024-03-13T08:11:16.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c8b87640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:16.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c8b87640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:16.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c8b87640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:16.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c9b89640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:16.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c9b89640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:16.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c9b89640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:16.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c9388640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:16.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c9388640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:16.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c9388640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:16.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c8b87640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:16.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c8b87640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:16.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.716+0000 7f58c8b87640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:16.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.719+0000 7f58c9b89640 10 monclient(hunting): handle_auth_done global_id 19717 payload 824 2024-03-13T08:11:16.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.719+0000 7f58c9b89640 1 -- 172.21.15.132:0/1558780331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c406fb50 msgr2=0x7f58c413a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:16.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.719+0000 7f58c9b89640 1 --2- 172.21.15.132:0/1558780331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c406fb50 0x7f58c413a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.719+0000 7f58c9b89640 1 -- 172.21.15.132:0/1558780331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58c4071cf0 msgr2=0x7f58c413ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:16.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.719+0000 7f58c9b89640 1 --2- 172.21.15.132:0/1558780331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58c4071cf0 0x7f58c413ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:16.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.719+0000 7f58c9b89640 10 monclient: _finish_hunting 0 2024-03-13T08:11:16.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.719+0000 7f58c9b89640 1 monclient: found mon.noname-b 2024-03-13T08:11:16.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.719+0000 7f58c9b89640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:16.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.719+0000 7f58c9b89640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T08:11:16.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.719+0000 7f58c9b89640 1 -- 172.21.15.132:0/1558780331 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c41d4180 con 0x7f58c413a090 2024-03-13T08:11:16.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.721+0000 7f58c9b89640 1 --2- 172.21.15.132:0/1558780331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58c413a090 0x7f58c4141490 secure :-1 s=READY pgs=4739 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009f70 tx=0x7f58bc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:16.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.721+0000 7f58b9ffb640 10 client.?.objecter ms_handle_connect 0x7f58c413a090 2024-03-13T08:11:16.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.721+0000 7f58b9ffb640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:16.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58b9ffb640 1 -- 172.21.15.132:0/1558780331 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc043020 con 0x7f58c413a090 2024-03-13T08:11:16.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58b9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:16.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58b9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:16.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58b9ffb640 10 monclient: dump: 2024-03-13T08:11:16.729 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:16.729 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:16.729 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:16.729 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:16.729 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:16.729 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:16.729 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:16.729 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:16.729 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:16.729 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:16.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58b9ffb640 10 monclient: _finish_auth 0 2024-03-13T08:11:16.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58b9ffb640 10 monclient: _check_auth_tickets 2024-03-13T08:11:16.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:16.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58b9ffb640 1 -- 172.21.15.132:0/1558780331 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58bc03e450 con 0x7f58c413a090 2024-03-13T08:11:16.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58cb5ec640 5 monclient: authenticate success, global_id 19717 2024-03-13T08:11:16.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58b9ffb640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:16.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58cb5ec640 10 monclient: _renew_subs 2024-03-13T08:11:16.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58b9ffb640 1 -- 172.21.15.132:0/1558780331 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc03e820 con 0x7f58c413a090 2024-03-13T08:11:16.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58cb5ec640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:16.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58cb5ec640 1 -- 172.21.15.132:0/1558780331 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c41d43b0 con 0x7f58c413a090 2024-03-13T08:11:16.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58b9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:16.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58b9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:16.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58bb7fe640 4 set_mon_vals no callback set 2024-03-13T08:11:16.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58b9ffb640 10 monclient: dump: 2024-03-13T08:11:16.730 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:16.731 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:16.731 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:16.731 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:16.731 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:16.731 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:16.731 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:16.731 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:16.731 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:16.731 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:16.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58cb5ec640 10 client.19717.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:16.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58cb5ec640 10 monclient: _renew_subs 2024-03-13T08:11:16.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58cb5ec640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:16.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58cb5ec640 1 -- 172.21.15.132:0/1558780331 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c41d6f90 con 0x7f58c413a090 2024-03-13T08:11:16.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.725+0000 7f58cb5ec640 1 librados: init done 2024-03-13T08:11:16.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.726+0000 7f58cb5ec640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:16.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.726+0000 7f58cb5ec640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:16.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.726+0000 7f58cb5ec640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:16.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.726+0000 7f58cb5ec640 1 -- 172.21.15.132:0/1558780331 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c41d3cb0 con 0x7f58c413a090 2024-03-13T08:11:16.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.744+0000 7f58b9ffb640 1 -- 172.21.15.132:0/1558780331 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f58bc046020 con 0x7f58c413a090 2024-03-13T08:11:16.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.744+0000 7f58b9ffb640 1 --2- 172.21.15.132:0/1558780331 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f58a806d590 0x7f58a806fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:16.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.744+0000 7f58c9388640 1 --2- 172.21.15.132:0/1558780331 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f58a806d590 0x7f58a806fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:16.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.744+0000 7f58c9388640 10 monclient: get_auth_request con 0x7f58a806d590 auth_method 0 2024-03-13T08:11:16.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.745+0000 7f58b9ffb640 10 client.19717.objecter ms_handle_connect 0x7f58a806d590 2024-03-13T08:11:16.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.745+0000 7f58c9388640 1 --2- 172.21.15.132:0/1558780331 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f58a806d590 0x7f58a806fa50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f58c0006fd0 tx=0x7f58c0009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:16.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.747+0000 7f58b9ffb640 1 -- 172.21.15.132:0/1558780331 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f58bc079d90 con 0x7f58c413a090 2024-03-13T08:11:16.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.747+0000 7f58b9ffb640 10 client.19717.objecter ms_dispatch 0x7f58c4000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:16.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.747+0000 7f58b9ffb640 3 client.19717.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:16.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.747+0000 7f58b9ffb640 3 client.19717.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:16.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.748+0000 7f58b9ffb640 20 client.19717.objecter dump_active .. 0 homeless 2024-03-13T08:11:16.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.995+0000 7f58b9ffb640 1 -- 172.21.15.132:0/1558780331 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f58bc04d070 con 0x7f58c413a090 2024-03-13T08:11:16.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.995+0000 7f58b9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:16.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:16.995+0000 7f58b9ffb640 10 monclient: _finish_command 1 = system:0 2024-03-13T08:11:17.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.179+0000 7f58cb5ec640 1 -- 172.21.15.132:0/1558780331 --> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f58c41d4900 con 0x7f58a806d590 2024-03-13T08:11:17.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.179+0000 7f58b9ffb640 1 -- 172.21.15.132:0/1558780331 <== mgr.4104 v2:172.21.15.132:6800/3847550149 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f58c41d4900 con 0x7f58a806d590 2024-03-13T08:11:17.180 INFO:teuthology.orchestra.run.smithi132.stderr:instructed osd(s) 5 to deep-scrub 2024-03-13T08:11:17.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.181+0000 7f58cb5ec640 10 librados: watch_flush enter 2024-03-13T08:11:17.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.181+0000 7f58cb5ec640 10 librados: watch_flush exit 2024-03-13T08:11:17.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.181+0000 7f58cb5ec640 20 client.19717.objecter shutdown clearing up homeless session... 2024-03-13T08:11:17.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.181+0000 7f58cb5ec640 10 client.19717.objecter successfully canceled tick 2024-03-13T08:11:17.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.181+0000 7f58cb5ec640 1 -- 172.21.15.132:0/1558780331 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f58a806d590 msgr2=0x7f58a806fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:17.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.181+0000 7f58cb5ec640 1 --2- 172.21.15.132:0/1558780331 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f58a806d590 0x7f58a806fa50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f58c0006fd0 tx=0x7f58c0009060 comp rx=0 tx=0).stop 2024-03-13T08:11:17.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.181+0000 7f58cb5ec640 10 monclient: shutdown 2024-03-13T08:11:17.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.181+0000 7f58cb5ec640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:17.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.181+0000 7f58cb5ec640 1 -- 172.21.15.132:0/1558780331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58c413a090 msgr2=0x7f58c4141490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:17.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.181+0000 7f58cb5ec640 1 --2- 172.21.15.132:0/1558780331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58c413a090 0x7f58c4141490 secure :-1 s=READY pgs=4739 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009f70 tx=0x7f58bc004a40 comp rx=0 tx=0).stop 2024-03-13T08:11:17.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.181+0000 7f58cb5ec640 1 -- 172.21.15.132:0/1558780331 shutdown_connections 2024-03-13T08:11:17.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.181+0000 7f58cb5ec640 1 --2- 172.21.15.132:0/1558780331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58c413a090 0x7f58c4141490 unknown :-1 s=CLOSED pgs=4739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:17.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.181+0000 7f58cb5ec640 1 --2- 172.21.15.132:0/1558780331 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f58a806d590 0x7f58a806fa50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:17.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.181+0000 7f58cb5ec640 1 --2- 172.21.15.132:0/1558780331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58c4071cf0 0x7f58c413ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:17.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.181+0000 7f58cb5ec640 1 --2- 172.21.15.132:0/1558780331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c406fb50 0x7f58c413a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:17.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.182+0000 7f58cb5ec640 1 -- 172.21.15.132:0/1558780331 >> 172.21.15.132:0/1558780331 conn(0x7f58c40b3e20 msgr2=0x7f58c413dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:17.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.182+0000 7f58cb5ec640 1 -- 172.21.15.132:0/1558780331 shutdown_connections 2024-03-13T08:11:17.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.182+0000 7f58cb5ec640 1 -- 172.21.15.132:0/1558780331 wait complete. 2024-03-13T08:11:17.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.182+0000 7f58cb5ec640 1 librados: shutdown 2024-03-13T08:11:17.196 INFO:tasks.ceph:Scrubbing osd.6 2024-03-13T08:11:17.196 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-13T08:11:17.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.305+0000 7f65cd4eb640 10 monclient: get_monmap_and_config 2024-03-13T08:11:17.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.305+0000 7f65cd4eb640 10 monclient: build_initial_monmap 2024-03-13T08:11:17.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.305+0000 7f65cd4eb640 10 monclient: monmap: 2024-03-13T08:11:17.306 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:17.306 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:17.306 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:17.305976+0000 2024-03-13T08:11:17.306 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:17.305976+0000 2024-03-13T08:11:17.306 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:17.306 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:17.306 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:17.306 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:17.307 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:17.307 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:17.307 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:17.307 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:17.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.305+0000 7f65cd4eb640 1 Processor -- start 2024-03-13T08:11:17.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.305+0000 7f65cd4eb640 1 -- start start 2024-03-13T08:11:17.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.305+0000 7f65cd4eb640 10 monclient: init 2024-03-13T08:11:17.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.306+0000 7f65cd4eb640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:17.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.306+0000 7f65cd4eb640 10 monclient: _add_conns ranks=[2,1,0,3,4] 2024-03-13T08:11:17.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.306+0000 7f65cd4eb640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65c80b52a0 0x7f65c813fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:17.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.306+0000 7f65cd4eb640 10 monclient(hunting): picked mon.noname-b con 0x7f65c80b52a0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:17.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.306+0000 7f65cd4eb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f65c80b24b0 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:17.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.306+0000 7f65cd4eb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f65c80a9ef0 addr v1:172.21.15.132:6789/0 2024-03-13T08:11:17.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.306+0000 7f65cd4eb640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c80b4790 con 0x7f65c80a9ef0 2024-03-13T08:11:17.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.306+0000 7f65cd4eb640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c813ff50 con 0x7f65c80b24b0 2024-03-13T08:11:17.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.306+0000 7f65cd4eb640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:17.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.306+0000 7f65cd4eb640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80b5680 con 0x7f65c80b52a0 2024-03-13T08:11:17.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.306+0000 7f65cd4eb640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:17.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.306+0000 7f65cd4eb640 10 monclient(hunting): authenticate will time out at 3504.714628s 2024-03-13T08:11:17.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.307+0000 7f65c77fe640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f65c80a9ef0 0x7f65c8146390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:50432/0 (socket says 172.21.15.132:50432) 2024-03-13T08:11:17.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.307+0000 7f65c77fe640 1 -- 172.21.15.132:0/3035422372 learned_addr learned my addr 172.21.15.132:0/3035422372 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T08:11:17.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.307+0000 7f65c6ffd640 1 --2- 172.21.15.132:0/3035422372 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65c80b52a0 0x7f65c813fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:17.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.307+0000 7f65c6ffd640 10 monclient(hunting): get_auth_request con 0x7f65c80b52a0 auth_method 0 2024-03-13T08:11:17.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.307+0000 7f65c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:17.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.307+0000 7f65c6ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:17.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.307+0000 7f65c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:17.314 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.308+0000 7f65c6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:17.315 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.308+0000 7f65c6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:17.315 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.308+0000 7f65c6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:17.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.309+0000 7f65c5ffb640 1 -- 172.21.15.132:0/3035422372 <== mon.0 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840299041 0 0) 0x7f65c80b4790 con 0x7f65c80a9ef0 2024-03-13T08:11:17.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.309+0000 7f65c5ffb640 10 monclient(hunting): _negotiate 2024-03-13T08:11:17.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.309+0000 7f65c5ffb640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:17.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.309+0000 7f65c5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:17.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.309+0000 7f65c5ffb640 10 monclient(hunting): my global_id is 16077 2024-03-13T08:11:17.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.309+0000 7f65c5ffb640 1 -- 172.21.15.132:0/3035422372 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65b0003520 con 0x7f65c80a9ef0 2024-03-13T08:11:17.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.309+0000 7f65c6ffd640 10 monclient(hunting): handle_auth_done global_id 19025 payload 274 2024-03-13T08:11:17.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.309+0000 7f65c6ffd640 1 -- 172.21.15.132:0/3035422372 >> v1:172.21.15.195:6789/0 conn(0x7f65c80b24b0 legacy=0x7f65c8145c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:17.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.309+0000 7f65c6ffd640 1 -- 172.21.15.132:0/3035422372 >> v1:172.21.15.132:6789/0 conn(0x7f65c80a9ef0 legacy=0x7f65c8146390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:17.320 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.309+0000 7f65c6ffd640 10 monclient: _finish_hunting 0 2024-03-13T08:11:17.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.309+0000 7f65c6ffd640 1 monclient: found mon.noname-b 2024-03-13T08:11:17.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.309+0000 7f65c5ffb640 1 -- 172.21.15.132:0/3035422372 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958733908 0 0) 0x7f65c813ff50 con 0x7f65c80b24b0 2024-03-13T08:11:17.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.309+0000 7f65c6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:17.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.309+0000 7f65c6ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T08:11:17.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.309+0000 7f65c6ffd640 1 -- 172.21.15.132:0/3035422372 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c8140120 con 0x7f65c80b52a0 2024-03-13T08:11:17.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.309+0000 7f65c5ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-03-13T08:11:17.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.311+0000 7f65c6ffd640 1 --2- 172.21.15.132:0/3035422372 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65c80b52a0 0x7f65c813fa10 secure :-1 s=READY pgs=3544 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0097d0 tx=0x7f65bc02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f888c059d1ffe25 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:17.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65c5ffb640 1 -- 172.21.15.132:0/3035422372 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc035020 con 0x7f65c80b52a0 2024-03-13T08:11:17.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65c5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:17.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65c5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:17.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65c5ffb640 10 monclient: dump: 2024-03-13T08:11:17.322 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:17.322 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:17.322 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:17.322 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:17.323 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:17.323 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:17.323 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:17.323 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:17.323 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:17.324 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:17.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65c5ffb640 10 monclient: _finish_auth 0 2024-03-13T08:11:17.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65c5ffb640 10 monclient: _check_auth_tickets 2024-03-13T08:11:17.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65c5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:17.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65c5ffb640 1 -- 172.21.15.132:0/3035422372 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc031800 con 0x7f65c80b52a0 2024-03-13T08:11:17.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65cd4eb640 5 monclient: authenticate success, global_id 19025 2024-03-13T08:11:17.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65cd4eb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T08:11:17.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65c5ffb640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:17.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65cd4eb640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:17.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65cd4eb640 4 set_mon_vals no callback set 2024-03-13T08:11:17.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65c5ffb640 1 -- 172.21.15.132:0/3035422372 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc031bd0 con 0x7f65c80b52a0 2024-03-13T08:11:17.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65c5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:17.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65c5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T08:11:17.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65c5ffb640 10 monclient: dump: 2024-03-13T08:11:17.325 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:17.325 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:17.325 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:17.325 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:17.325 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:17.325 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:17.325 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:17.325 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:17.325 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:17.325 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:17.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65cd4eb640 10 monclient: shutdown 2024-03-13T08:11:17.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65cd4eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:17.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65cd4eb640 1 -- 172.21.15.132:0/3035422372 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65c80b52a0 msgr2=0x7f65c813fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:17.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65cd4eb640 1 --2- 172.21.15.132:0/3035422372 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65c80b52a0 0x7f65c813fa10 secure :-1 s=READY pgs=3544 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0097d0 tx=0x7f65bc02b8a0 comp rx=0 tx=0).stop 2024-03-13T08:11:17.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65cd4eb640 1 -- 172.21.15.132:0/3035422372 shutdown_connections 2024-03-13T08:11:17.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65cd4eb640 1 --2- 172.21.15.132:0/3035422372 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65c80b52a0 0x7f65c813fa10 unknown :-1 s=CLOSED pgs=3544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:17.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65cd4eb640 1 -- 172.21.15.132:0/3035422372 >> 172.21.15.132:0/3035422372 conn(0x7f65c8066e20 msgr2=0x7f65c8067230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:17.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65cd4eb640 1 -- 172.21.15.132:0/3035422372 shutdown_connections 2024-03-13T08:11:17.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.315+0000 7f65cd4eb640 1 -- 172.21.15.132:0/3035422372 wait complete. 2024-03-13T08:11:17.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.316+0000 7f65cd4eb640 10 monclient: build_initial_monmap 2024-03-13T08:11:17.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.316+0000 7f65cd4eb640 10 monclient: monmap: 2024-03-13T08:11:17.326 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:17.326 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:17.326 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:17.326 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:17.327 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:17.327 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:17.327 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:17.327 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:17.327 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:17.327 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:17.327 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 1 librados: starting msgr at 2024-03-13T08:11:17.327 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 1 librados: starting objecter 2024-03-13T08:11:17.327 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 1 Processor -- start 2024-03-13T08:11:17.327 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 1 -- start start 2024-03-13T08:11:17.327 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 1 librados: setting wanted keys 2024-03-13T08:11:17.327 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 1 librados: calling monclient init 2024-03-13T08:11:17.327 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 10 monclient: init 2024-03-13T08:11:17.327 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:17.327 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 10 monclient: _add_conns ranks=[2,1,0] 2024-03-13T08:11:17.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65c80a9ef0 0x7f65c80b87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:17.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 10 monclient(hunting): picked mon.noname-c con 0x7f65c80a9ef0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:17.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65c80b24b0 0x7f65c80b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:17.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 10 monclient(hunting): picked mon.noname-b con 0x7f65c80b24b0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:17.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65c80b52a0 0x7f65c80b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:17.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 10 monclient(hunting): picked mon.noname-a con 0x7f65c80b52a0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:17.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:17.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c814e1d0 con 0x7f65c80b52a0 2024-03-13T08:11:17.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:17.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80aa4b0 con 0x7f65c80b24b0 2024-03-13T08:11:17.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:17.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c814e350 con 0x7f65c80a9ef0 2024-03-13T08:11:17.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:17.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.317+0000 7f65cd4eb640 10 monclient(hunting): authenticate will time out at 3504.726010s 2024-03-13T08:11:17.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c6ffd640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65c80a9ef0 0x7f65c80b87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:17.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c77fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65c80b52a0 0x7f65c80b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:17.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c67fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65c80b24b0 0x7f65c80b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:17.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c6ffd640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65c80a9ef0 0x7f65c80b87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:59866/0 (socket says 172.21.15.132:59866) 2024-03-13T08:11:17.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c77fe640 10 monclient(hunting): get_auth_request con 0x7f65c80b52a0 auth_method 0 2024-03-13T08:11:17.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c6ffd640 1 -- 172.21.15.132:0/1079484535 learned_addr learned my addr 172.21.15.132:0/1079484535 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:17.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:17.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c77fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:17.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:17.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c6ffd640 10 monclient(hunting): get_auth_request con 0x7f65c80a9ef0 auth_method 0 2024-03-13T08:11:17.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:17.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c6ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:17.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:17.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c67fc640 10 monclient(hunting): get_auth_request con 0x7f65c80b24b0 auth_method 0 2024-03-13T08:11:17.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:17.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c67fc640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:17.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.318+0000 7f65c67fc640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:17.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.319+0000 7f65c67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:17.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.319+0000 7f65c67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:17.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.319+0000 7f65c67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:17.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.319+0000 7f65c6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:17.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.319+0000 7f65c6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:17.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.319+0000 7f65c6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:17.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.319+0000 7f65c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:17.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.319+0000 7f65c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:17.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.319+0000 7f65c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:17.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.321+0000 7f65c67fc640 10 monclient(hunting): handle_auth_done global_id 19723 payload 824 2024-03-13T08:11:17.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.321+0000 7f65c67fc640 1 -- 172.21.15.132:0/1079484535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65c80a9ef0 msgr2=0x7f65c80b87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:17.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.321+0000 7f65c67fc640 1 --2- 172.21.15.132:0/1079484535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65c80a9ef0 0x7f65c80b87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:17.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.321+0000 7f65c67fc640 1 -- 172.21.15.132:0/1079484535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65c80b52a0 msgr2=0x7f65c80b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:17.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.321+0000 7f65c67fc640 1 --2- 172.21.15.132:0/1079484535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65c80b52a0 0x7f65c80b6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:17.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.321+0000 7f65c67fc640 10 monclient: _finish_hunting 0 2024-03-13T08:11:17.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.321+0000 7f65c67fc640 1 monclient: found mon.noname-b 2024-03-13T08:11:17.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.321+0000 7f65c67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:17.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.321+0000 7f65c67fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T08:11:17.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.321+0000 7f65c67fc640 1 -- 172.21.15.132:0/1079484535 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80b8000 con 0x7f65c80b24b0 2024-03-13T08:11:17.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.324+0000 7f65c67fc640 1 --2- 172.21.15.132:0/1079484535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65c80b24b0 0x7f65c80b5bd0 secure :-1 s=READY pgs=4741 cs=0 l=1 rev1=1 crypto rx=0x7f65c80b28b0 tx=0x7f65b800c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:17.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.324+0000 7f65af7fe640 10 client.?.objecter ms_handle_connect 0x7f65c80b24b0 2024-03-13T08:11:17.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.324+0000 7f65af7fe640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:17.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65af7fe640 1 -- 172.21.15.132:0/1079484535 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b8017070 con 0x7f65c80b24b0 2024-03-13T08:11:17.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65af7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:17.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65af7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:17.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65af7fe640 10 monclient: dump: 2024-03-13T08:11:17.335 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:17.335 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:17.335 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:17.335 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:17.335 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:17.335 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:17.335 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:17.335 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:17.335 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:17.335 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:17.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65af7fe640 10 monclient: _finish_auth 0 2024-03-13T08:11:17.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65af7fe640 10 monclient: _check_auth_tickets 2024-03-13T08:11:17.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65af7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:17.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65af7fe640 1 -- 172.21.15.132:0/1079484535 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b80035f0 con 0x7f65c80b24b0 2024-03-13T08:11:17.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65af7fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:17.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65af7fe640 1 -- 172.21.15.132:0/1079484535 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b80039e0 con 0x7f65c80b24b0 2024-03-13T08:11:17.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65af7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:17.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65af7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:17.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65af7fe640 10 monclient: dump: 2024-03-13T08:11:17.336 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:17.336 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:17.336 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:17.336 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:17.336 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:17.336 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:17.337 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:17.337 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:17.337 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:17.337 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:17.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65cd4eb640 5 monclient: authenticate success, global_id 19723 2024-03-13T08:11:17.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65cd4eb640 10 monclient: _renew_subs 2024-03-13T08:11:17.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65cd4eb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:17.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65cd4eb640 1 -- 172.21.15.132:0/1079484535 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c80b7be0 con 0x7f65c80b24b0 2024-03-13T08:11:17.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65c4ff9640 4 set_mon_vals no callback set 2024-03-13T08:11:17.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65cd4eb640 10 client.19723.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:17.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65cd4eb640 10 monclient: _renew_subs 2024-03-13T08:11:17.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65cd4eb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:17.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65cd4eb640 1 -- 172.21.15.132:0/1079484535 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c80b6650 con 0x7f65c80b24b0 2024-03-13T08:11:17.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.327+0000 7f65cd4eb640 1 librados: init done 2024-03-13T08:11:17.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.328+0000 7f65cd4eb640 10 client.19723.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:17.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.328+0000 7f65cd4eb640 15 client.19723.objecter _session_command_op_assign -1 1 2024-03-13T08:11:17.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.328+0000 7f65cd4eb640 20 client.19723.objecter _get_session osd=-1 returning homeless 2024-03-13T08:11:17.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.328+0000 7f65cd4eb640 10 client.19723.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:17.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.328+0000 7f65cd4eb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:17.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.328+0000 7f65cd4eb640 1 -- 172.21.15.132:0/1079484535 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65c804f000 con 0x7f65c80b24b0 2024-03-13T08:11:17.349 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.348+0000 7f65af7fe640 1 -- 172.21.15.132:0/1079484535 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f65b801a070 con 0x7f65c80b24b0 2024-03-13T08:11:17.349 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.348+0000 7f65af7fe640 1 --2- 172.21.15.132:0/1079484535 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f65b406d560 0x7f65b406fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:17.349 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.348+0000 7f65c6ffd640 1 --2- 172.21.15.132:0/1079484535 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f65b406d560 0x7f65b406fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:17.349 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.348+0000 7f65c6ffd640 10 monclient: get_auth_request con 0x7f65b406d560 auth_method 0 2024-03-13T08:11:17.349 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.349+0000 7f65af7fe640 10 client.19723.objecter ms_handle_connect 0x7f65b406d560 2024-03-13T08:11:17.349 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.349+0000 7f65c6ffd640 1 --2- 172.21.15.132:0/1079484535 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f65b406d560 0x7f65b406fa20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f65bc02b8e0 tx=0x7f65bc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:17.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.352+0000 7f65af7fe640 1 -- 172.21.15.132:0/1079484535 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f65b80569c0 con 0x7f65c80b24b0 2024-03-13T08:11:17.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.352+0000 7f65af7fe640 10 client.19723.objecter ms_dispatch 0x7f65c8000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:17.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.352+0000 7f65af7fe640 3 client.19723.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:17.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.353+0000 7f65af7fe640 3 client.19723.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:17.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.353+0000 7f65af7fe640 10 client.19723.objecter checking command 1 2024-03-13T08:11:17.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.353+0000 7f65af7fe640 1 --2- 172.21.15.132:0/1079484535 >> [v2:172.21.15.195:6800/4106887608,v1:172.21.15.195:6801/4106887608] conn(0x7f65b40751d0 0x7f65b4077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:17.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.353+0000 7f65af7fe640 20 client.19723.objecter _get_session s=0x7f65b4074b00 osd=6 3 2024-03-13T08:11:17.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.353+0000 7f65af7fe640 20 client.19723.objecter put_session s=0x7f65b4074b00 osd=6 3 2024-03-13T08:11:17.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.353+0000 7f65af7fe640 15 client.19723.objecter _session_command_op_remove -1 1 2024-03-13T08:11:17.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.353+0000 7f65af7fe640 20 client.19723.objecter _get_session s=0x7f65b4074b00 osd=6 3 2024-03-13T08:11:17.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.353+0000 7f65af7fe640 20 client.19723.objecter get_session s=0x7f65b4074b00 osd=6 3 2024-03-13T08:11:17.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.353+0000 7f65af7fe640 15 client.19723.objecter _session_command_op_assign 6 1 2024-03-13T08:11:17.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.353+0000 7f65af7fe640 20 client.19723.objecter put_session s=0x7f65b4074b00 osd=6 4 2024-03-13T08:11:17.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.353+0000 7f65af7fe640 10 client.19723.objecter _send_command 1 2024-03-13T08:11:17.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.353+0000 7f65af7fe640 1 -- 172.21.15.132:0/1079484535 --> [v2:172.21.15.195:6800/4106887608,v1:172.21.15.195:6801/4106887608] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65b801be50 con 0x7f65b40751d0 2024-03-13T08:11:17.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.353+0000 7f65af7fe640 20 client.19723.objecter dump_active .. 0 homeless 2024-03-13T08:11:17.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.353+0000 7f65c77fe640 1 --2- 172.21.15.132:0/1079484535 >> [v2:172.21.15.195:6800/4106887608,v1:172.21.15.195:6801/4106887608] conn(0x7f65b40751d0 0x7f65b4077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:17.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.354+0000 7f65c77fe640 10 monclient: get_auth_request con 0x7f65b40751d0 auth_method 0 2024-03-13T08:11:17.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.355+0000 7f65af7fe640 1 -- 172.21.15.132:0/1079484535 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=1293) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65b8054bc0 con 0x7f65c80b24b0 2024-03-13T08:11:17.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.355+0000 7f65af7fe640 10 monclient: handle_get_version_reply finishing 1 version 1293 2024-03-13T08:11:17.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.356+0000 7f65c77fe640 1 --2- 172.21.15.132:0/1079484535 >> [v2:172.21.15.195:6800/4106887608,v1:172.21.15.195:6801/4106887608] conn(0x7f65b40751d0 0x7f65b4077670 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:17.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.356+0000 7f65af7fe640 10 client.19723.objecter ms_handle_connect 0x7f65b40751d0 2024-03-13T08:11:17.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.380+0000 7f65af7fe640 1 -- 172.21.15.132:0/1079484535 <== osd.6 v2:172.21.15.195:6800/4106887608 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f65b801be50 con 0x7f65b40751d0 2024-03-13T08:11:17.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.380+0000 7f65af7fe640 10 client.19723.objecter ms_dispatch 0x7f65c8000b90 command_reply(tid 1: 0 ) v1 2024-03-13T08:11:17.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.380+0000 7f65af7fe640 10 client.19723.objecter _finish_command 1 = system:0 2024-03-13T08:11:17.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.380+0000 7f65af7fe640 20 client.19723.objecter put_session s=0x7f65b4074b00 osd=6 4 2024-03-13T08:11:17.381 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.380+0000 7f65af7fe640 15 client.19723.objecter _session_command_op_remove 6 1 2024-03-13T08:11:17.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.394+0000 7f65cd4eb640 10 client.19723.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-03-13T08:11:17.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.394+0000 7f65cd4eb640 15 client.19723.objecter _session_command_op_assign -1 2 2024-03-13T08:11:17.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.394+0000 7f65cd4eb640 20 client.19723.objecter _get_session s=0x7f65b4074b00 osd=6 3 2024-03-13T08:11:17.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.394+0000 7f65cd4eb640 20 client.19723.objecter put_session s=0x7f65b4074b00 osd=6 3 2024-03-13T08:11:17.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.394+0000 7f65cd4eb640 20 client.19723.objecter _get_session s=0x7f65b4074b00 osd=6 3 2024-03-13T08:11:17.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.394+0000 7f65cd4eb640 15 client.19723.objecter _session_command_op_remove -1 2 2024-03-13T08:11:17.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.394+0000 7f65cd4eb640 20 client.19723.objecter get_session s=0x7f65b4074b00 osd=6 3 2024-03-13T08:11:17.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.394+0000 7f65cd4eb640 15 client.19723.objecter _session_command_op_assign 6 2 2024-03-13T08:11:17.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.394+0000 7f65cd4eb640 20 client.19723.objecter put_session s=0x7f65b4074b00 osd=6 4 2024-03-13T08:11:17.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.394+0000 7f65cd4eb640 10 client.19723.objecter _send_command 2 2024-03-13T08:11:17.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.394+0000 7f65cd4eb640 1 -- 172.21.15.132:0/1079484535 --> [v2:172.21.15.195:6800/4106887608,v1:172.21.15.195:6801/4106887608] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f65c81daa10 con 0x7f65b40751d0 2024-03-13T08:11:17.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.407+0000 7f65af7fe640 1 -- 172.21.15.132:0/1079484535 <== osd.6 v2:172.21.15.195:6800/4106887608 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f65c81daa10 con 0x7f65b40751d0 2024-03-13T08:11:17.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.407+0000 7f65af7fe640 10 client.19723.objecter ms_dispatch 0x7f65c8000b90 command_reply(tid 2: 0 ) v1 2024-03-13T08:11:17.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.407+0000 7f65af7fe640 10 client.19723.objecter _finish_command 2 = system:0 2024-03-13T08:11:17.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.407+0000 7f65af7fe640 20 client.19723.objecter put_session s=0x7f65b4074b00 osd=6 4 2024-03-13T08:11:17.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.407+0000 7f65af7fe640 15 client.19723.objecter _session_command_op_remove 6 2 2024-03-13T08:11:17.409 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-13T08:11:17.409 INFO:teuthology.orchestra.run.smithi132.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-13T08:11:17.409 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-13T08:11:17.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 10 librados: watch_flush enter 2024-03-13T08:11:17.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 10 librados: watch_flush exit 2024-03-13T08:11:17.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 10 client.19723.objecter close_session for osd.6 2024-03-13T08:11:17.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 1 -- 172.21.15.132:0/1079484535 >> [v2:172.21.15.195:6800/4106887608,v1:172.21.15.195:6801/4106887608] conn(0x7f65b40751d0 msgr2=0x7f65b4077670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:17.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 1 --2- 172.21.15.132:0/1079484535 >> [v2:172.21.15.195:6800/4106887608,v1:172.21.15.195:6801/4106887608] conn(0x7f65b40751d0 0x7f65b4077670 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:17.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 20 client.19723.objecter put_session s=0x7f65b4074b00 osd=6 1 2024-03-13T08:11:17.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 20 client.19723.objecter shutdown clearing up homeless session... 2024-03-13T08:11:17.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 10 client.19723.objecter successfully canceled tick 2024-03-13T08:11:17.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 1 -- 172.21.15.132:0/1079484535 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f65b406d560 msgr2=0x7f65b406fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:17.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 1 --2- 172.21.15.132:0/1079484535 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f65b406d560 0x7f65b406fa20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f65bc02b8e0 tx=0x7f65bc008030 comp rx=0 tx=0).stop 2024-03-13T08:11:17.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 10 monclient: shutdown 2024-03-13T08:11:17.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:17.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 1 -- 172.21.15.132:0/1079484535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65c80b24b0 msgr2=0x7f65c80b5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:17.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 1 --2- 172.21.15.132:0/1079484535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65c80b24b0 0x7f65c80b5bd0 secure :-1 s=READY pgs=4741 cs=0 l=1 rev1=1 crypto rx=0x7f65c80b28b0 tx=0x7f65b800c890 comp rx=0 tx=0).stop 2024-03-13T08:11:17.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65c77fe640 1 -- 172.21.15.132:0/1079484535 reap_dead start 2024-03-13T08:11:17.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 1 -- 172.21.15.132:0/1079484535 shutdown_connections 2024-03-13T08:11:17.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 1 -- 172.21.15.132:0/1079484535 >> 172.21.15.132:0/1079484535 conn(0x7f65c8066e20 msgr2=0x7f65c80b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:17.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 1 -- 172.21.15.132:0/1079484535 shutdown_connections 2024-03-13T08:11:17.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 1 -- 172.21.15.132:0/1079484535 wait complete. 2024-03-13T08:11:17.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.408+0000 7f65cd4eb640 1 librados: shutdown 2024-03-13T08:11:17.420 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-13T08:11:17.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.520+0000 7f5334445640 10 monclient: get_monmap_and_config 2024-03-13T08:11:17.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.520+0000 7f5334445640 10 monclient: build_initial_monmap 2024-03-13T08:11:17.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.520+0000 7f5334445640 10 monclient: monmap: 2024-03-13T08:11:17.523 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:17.523 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:17.523 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:17.521081+0000 2024-03-13T08:11:17.523 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:17.521081+0000 2024-03-13T08:11:17.523 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:17.523 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:17.523 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:17.523 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:17.523 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:17.524 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:17.524 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:17.524 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:17.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.520+0000 7f5334445640 1 Processor -- start 2024-03-13T08:11:17.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.520+0000 7f5334445640 1 -- start start 2024-03-13T08:11:17.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.520+0000 7f5334445640 10 monclient: init 2024-03-13T08:11:17.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.521+0000 7f5334445640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:17.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.521+0000 7f5334445640 10 monclient: _add_conns ranks=[4,1,0,2,3] 2024-03-13T08:11:17.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.521+0000 7f5334445640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f532c06c800 0x7f532c142f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:17.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.521+0000 7f5334445640 10 monclient(hunting): picked mon.noname-c con 0x7f532c06c800 addr v2:172.21.15.195:3300/0 2024-03-13T08:11:17.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.521+0000 7f5334445640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f532c1434c0 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:17.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.521+0000 7f5334445640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f532c0af170 addr v1:172.21.15.132:6789/0 2024-03-13T08:11:17.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.521+0000 7f5334445640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532c0ae080 con 0x7f532c0af170 2024-03-13T08:11:17.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.521+0000 7f5334445640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532c0ae360 con 0x7f532c1434c0 2024-03-13T08:11:17.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.521+0000 7f5334445640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:17.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.521+0000 7f5334445640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532c0b0280 con 0x7f532c06c800 2024-03-13T08:11:17.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.521+0000 7f5334445640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:17.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.521+0000 7f5334445640 10 monclient(hunting): authenticate will time out at 3504.929781s 2024-03-13T08:11:17.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.522+0000 7f53329e2640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f532c0af170 0x7f532c14d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:50446/0 (socket says 172.21.15.132:50446) 2024-03-13T08:11:17.525 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.522+0000 7f53329e2640 1 -- 172.21.15.132:0/2848058665 learned_addr learned my addr 172.21.15.132:0/2848058665 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T08:11:17.526 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.522+0000 7f53321e1640 1 --2- 172.21.15.132:0/2848058665 >> v2:172.21.15.195:3300/0 conn(0x7f532c06c800 0x7f532c142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:17.526 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.522+0000 7f53321e1640 10 monclient(hunting): get_auth_request con 0x7f532c06c800 auth_method 0 2024-03-13T08:11:17.526 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.522+0000 7f53321e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:17.526 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.522+0000 7f53321e1640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:17.526 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.522+0000 7f53321e1640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:17.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.523+0000 7f53321e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:17.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.523+0000 7f53321e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:17.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.523+0000 7f53321e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:17.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.524+0000 7f53311df640 1 -- 172.21.15.132:0/2848058665 <== mon.0 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206123822 0 0) 0x7f532c0ae080 con 0x7f532c0af170 2024-03-13T08:11:17.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.524+0000 7f53311df640 10 monclient(hunting): _negotiate 2024-03-13T08:11:17.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.524+0000 7f53311df640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:17.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.524+0000 7f53311df640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:17.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.524+0000 7f53311df640 10 monclient(hunting): my global_id is 16083 2024-03-13T08:11:17.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.524+0000 7f53311df640 1 -- 172.21.15.132:0/2848058665 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5318003520 con 0x7f532c0af170 2024-03-13T08:11:17.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.524+0000 7f53321e1640 10 monclient(hunting): handle_auth_done global_id 19726 payload 274 2024-03-13T08:11:17.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.524+0000 7f53321e1640 1 -- 172.21.15.132:0/2848058665 >> v1:172.21.15.195:6789/0 conn(0x7f532c1434c0 legacy=0x7f532c0aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:17.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.524+0000 7f53321e1640 1 -- 172.21.15.132:0/2848058665 >> v1:172.21.15.132:6789/0 conn(0x7f532c0af170 legacy=0x7f532c14d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:17.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.524+0000 7f53321e1640 10 monclient: _finish_hunting 0 2024-03-13T08:11:17.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.524+0000 7f53321e1640 1 monclient: found mon.noname-c 2024-03-13T08:11:17.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.524+0000 7f53321e1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:17.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.524+0000 7f53321e1640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T08:11:17.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.524+0000 7f53321e1640 1 -- 172.21.15.132:0/2848058665 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c06d630 con 0x7f532c06c800 2024-03-13T08:11:17.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.526+0000 7f53321e1640 1 --2- 172.21.15.132:0/2848058665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532c06c800 0x7f532c142f80 secure :-1 s=READY pgs=4742 cs=0 l=1 rev1=1 crypto rx=0x7f531c0097a0 tx=0x7f531c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5c34c277a6f018d server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:17.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f53311df640 1 -- 172.21.15.132:0/2848058665 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c0315a0 con 0x7f532c06c800 2024-03-13T08:11:17.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f53311df640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:17.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f53311df640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T08:11:17.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f53311df640 10 monclient: dump: 2024-03-13T08:11:17.538 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:17.538 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:17.538 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:17.538 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:17.538 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:17.539 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:17.539 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:17.539 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:17.539 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:17.539 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:17.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f53311df640 10 monclient: _finish_auth 0 2024-03-13T08:11:17.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f53311df640 10 monclient: _check_auth_tickets 2024-03-13T08:11:17.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f53311df640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:17.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f53311df640 1 -- 172.21.15.132:0/2848058665 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531c037070 con 0x7f532c06c800 2024-03-13T08:11:17.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f53311df640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:17.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f53311df640 1 -- 172.21.15.132:0/2848058665 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c031960 con 0x7f532c06c800 2024-03-13T08:11:17.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f5334445640 5 monclient: authenticate success, global_id 19726 2024-03-13T08:11:17.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f5334445640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:17.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f5334445640 4 set_mon_vals no callback set 2024-03-13T08:11:17.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f53311df640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:17.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f53311df640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:17.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f53311df640 10 monclient: dump: 2024-03-13T08:11:17.540 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:17.540 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:17.540 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:17.541 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:17.541 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:17.541 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:17.541 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:17.541 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:17.541 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:17.541 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:17.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f5334445640 10 monclient: shutdown 2024-03-13T08:11:17.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f5334445640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:17.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f5334445640 1 -- 172.21.15.132:0/2848058665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532c06c800 msgr2=0x7f532c142f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:17.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f5334445640 1 --2- 172.21.15.132:0/2848058665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532c06c800 0x7f532c142f80 secure :-1 s=READY pgs=4742 cs=0 l=1 rev1=1 crypto rx=0x7f531c0097a0 tx=0x7f531c02b840 comp rx=0 tx=0).stop 2024-03-13T08:11:17.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f5334445640 1 -- 172.21.15.132:0/2848058665 shutdown_connections 2024-03-13T08:11:17.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f5334445640 1 --2- 172.21.15.132:0/2848058665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532c06c800 0x7f532c142f80 unknown :-1 s=CLOSED pgs=4742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:17.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.531+0000 7f5334445640 1 -- 172.21.15.132:0/2848058665 >> 172.21.15.132:0/2848058665 conn(0x7f532c0b5d70 msgr2=0x7f532c0b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:17.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.532+0000 7f5334445640 1 -- 172.21.15.132:0/2848058665 shutdown_connections 2024-03-13T08:11:17.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.532+0000 7f5334445640 1 -- 172.21.15.132:0/2848058665 wait complete. 2024-03-13T08:11:17.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.532+0000 7f5334445640 10 monclient: build_initial_monmap 2024-03-13T08:11:17.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.532+0000 7f5334445640 10 monclient: monmap: 2024-03-13T08:11:17.542 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:17.542 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:17.542 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:17.542 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:17.542 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:17.542 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:17.542 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:17.543 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:17.543 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:17.543 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:17.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.532+0000 7f5334445640 1 librados: starting msgr at 2024-03-13T08:11:17.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.532+0000 7f5334445640 1 librados: starting objecter 2024-03-13T08:11:17.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.532+0000 7f5334445640 1 Processor -- start 2024-03-13T08:11:17.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 1 -- start start 2024-03-13T08:11:17.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 1 librados: setting wanted keys 2024-03-13T08:11:17.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 1 librados: calling monclient init 2024-03-13T08:11:17.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 10 monclient: init 2024-03-13T08:11:17.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:17.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 10 monclient: _add_conns ranks=[2,1,0] 2024-03-13T08:11:17.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f532c06c800 0x7f532c124b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:17.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 10 monclient(hunting): picked mon.noname-c con 0x7f532c06c800 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:17.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532c0af170 0x7f532c1d7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:17.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 10 monclient(hunting): picked mon.noname-b con 0x7f532c0af170 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:17.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532c1434c0 0x7f532c1d8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:17.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 10 monclient(hunting): picked mon.noname-a con 0x7f532c1434c0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:17.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:17.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c06dc00 con 0x7f532c1434c0 2024-03-13T08:11:17.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:17.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c06da80 con 0x7f532c0af170 2024-03-13T08:11:17.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:17.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c0af5e0 con 0x7f532c06c800 2024-03-13T08:11:17.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:17.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.533+0000 7f5334445640 10 monclient(hunting): authenticate will time out at 3504.941703s 2024-03-13T08:11:17.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53329e2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532c1434c0 0x7f532c1d8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:17.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53321e1640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f532c06c800 0x7f532c124b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:17.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53319e0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532c0af170 0x7f532c1d7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:17.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53329e2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532c1434c0 0x7f532c1d8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56212/0 (socket says 172.21.15.132:56212) 2024-03-13T08:11:17.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53329e2640 1 -- 172.21.15.132:0/4098352172 learned_addr learned my addr 172.21.15.132:0/4098352172 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:17.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53329e2640 10 monclient(hunting): get_auth_request con 0x7f532c1434c0 auth_method 0 2024-03-13T08:11:17.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53329e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:17.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53329e2640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:17.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53329e2640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:17.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53319e0640 10 monclient(hunting): get_auth_request con 0x7f532c0af170 auth_method 0 2024-03-13T08:11:17.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53319e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:17.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53319e0640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:17.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53319e0640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:17.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53321e1640 10 monclient(hunting): get_auth_request con 0x7f532c06c800 auth_method 0 2024-03-13T08:11:17.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53321e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:17.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53321e1640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:17.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53321e1640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:17.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53329e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:17.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53329e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:17.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53329e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:17.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53319e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:17.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53319e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:17.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.534+0000 7f53319e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:17.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.535+0000 7f53321e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:17.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.535+0000 7f53321e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:17.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.535+0000 7f53321e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:17.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.537+0000 7f53319e0640 10 monclient(hunting): handle_auth_done global_id 19729 payload 824 2024-03-13T08:11:17.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.537+0000 7f53319e0640 1 -- 172.21.15.132:0/4098352172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f532c06c800 msgr2=0x7f532c124b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:17.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.537+0000 7f53319e0640 1 --2- 172.21.15.132:0/4098352172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f532c06c800 0x7f532c124b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:17.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.537+0000 7f53319e0640 1 -- 172.21.15.132:0/4098352172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532c1434c0 msgr2=0x7f532c1d8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:17.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.537+0000 7f53319e0640 1 --2- 172.21.15.132:0/4098352172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532c1434c0 0x7f532c1d8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:17.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.537+0000 7f53319e0640 10 monclient: _finish_hunting 0 2024-03-13T08:11:17.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.537+0000 7f53319e0640 1 monclient: found mon.noname-b 2024-03-13T08:11:17.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.537+0000 7f53319e0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:17.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.537+0000 7f53319e0640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.195:3300/0 2024-03-13T08:11:17.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.537+0000 7f53319e0640 1 -- 172.21.15.132:0/4098352172 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c1267f0 con 0x7f532c0af170 2024-03-13T08:11:17.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.540+0000 7f53319e0640 1 --2- 172.21.15.132:0/4098352172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532c0af170 0x7f532c1d7f20 secure :-1 s=READY pgs=4743 cs=0 l=1 rev1=1 crypto rx=0x7f532800b410 tx=0x7f532800b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:17.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.540+0000 7f5322ffd640 10 client.?.objecter ms_handle_connect 0x7f532c0af170 2024-03-13T08:11:17.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.540+0000 7f5322ffd640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:17.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5322ffd640 1 -- 172.21.15.132:0/4098352172 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328015020 con 0x7f532c0af170 2024-03-13T08:11:17.553 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5322ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:17.554 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5322ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:17.554 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5322ffd640 10 monclient: dump: 2024-03-13T08:11:17.554 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:17.554 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:17.554 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:17.554 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:17.554 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:17.555 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:17.555 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:17.555 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:17.555 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:17.555 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:17.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5322ffd640 10 monclient: _finish_auth 0 2024-03-13T08:11:17.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5322ffd640 10 monclient: _check_auth_tickets 2024-03-13T08:11:17.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5322ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:17.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5322ffd640 1 -- 172.21.15.132:0/4098352172 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5328011830 con 0x7f532c0af170 2024-03-13T08:11:17.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5334445640 5 monclient: authenticate success, global_id 19729 2024-03-13T08:11:17.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5334445640 10 monclient: _renew_subs 2024-03-13T08:11:17.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5334445640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:17.556 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5334445640 1 -- 172.21.15.132:0/4098352172 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532c124670 con 0x7f532c0af170 2024-03-13T08:11:17.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5322ffd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:17.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5322ffd640 1 -- 172.21.15.132:0/4098352172 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328011c60 con 0x7f532c0af170 2024-03-13T08:11:17.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5322ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:17.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5322ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:17.557 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5322ffd640 10 monclient: dump: 2024-03-13T08:11:17.557 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:17.557 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:17.558 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:17.558 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:17.558 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:17.558 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:17.558 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:17.558 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:17.559 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:17.559 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:17.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f53309de640 4 set_mon_vals no callback set 2024-03-13T08:11:17.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5334445640 10 client.19729.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:17.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5334445640 10 monclient: _renew_subs 2024-03-13T08:11:17.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5334445640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:17.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5334445640 1 -- 172.21.15.132:0/4098352172 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c125560 con 0x7f532c0af170 2024-03-13T08:11:17.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.543+0000 7f5334445640 1 librados: init done 2024-03-13T08:11:17.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.544+0000 7f5334445640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:17.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.544+0000 7f5334445640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:17.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.544+0000 7f5334445640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.195:3300/0 2024-03-13T08:11:17.560 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.544+0000 7f5334445640 1 -- 172.21.15.132:0/4098352172 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532c04f000 con 0x7f532c0af170 2024-03-13T08:11:17.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.809+0000 7f5322ffd640 1 -- 172.21.15.132:0/4098352172 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f5328018020 con 0x7f532c0af170 2024-03-13T08:11:17.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.809+0000 7f5322ffd640 1 --2- 172.21.15.132:0/4098352172 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f531006d5b0 0x7f531006fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:17.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.809+0000 7f53321e1640 1 --2- 172.21.15.132:0/4098352172 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f531006d5b0 0x7f531006fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:17.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.809+0000 7f53321e1640 10 monclient: get_auth_request con 0x7f531006d5b0 auth_method 0 2024-03-13T08:11:17.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.810+0000 7f5322ffd640 10 client.19729.objecter ms_handle_connect 0x7f531006d5b0 2024-03-13T08:11:17.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.810+0000 7f53321e1640 1 --2- 172.21.15.132:0/4098352172 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f531006d5b0 0x7f531006fa70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f531c036000 tx=0x7f531c03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:17.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.812+0000 7f5322ffd640 1 -- 172.21.15.132:0/4098352172 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f5328011e00 con 0x7f532c0af170 2024-03-13T08:11:17.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.812+0000 7f5322ffd640 10 client.19729.objecter ms_dispatch 0x7f532c000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:17.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.812+0000 7f5322ffd640 3 client.19729.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:17.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.812+0000 7f5322ffd640 3 client.19729.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:17.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.812+0000 7f5322ffd640 20 client.19729.objecter dump_active .. 0 homeless 2024-03-13T08:11:17.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.822+0000 7f5322ffd640 1 -- 172.21.15.132:0/4098352172 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f532801f030 con 0x7f532c0af170 2024-03-13T08:11:17.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.822+0000 7f5322ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:17.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.822+0000 7f5322ffd640 10 monclient: _finish_command 1 = system:0 2024-03-13T08:11:17.997 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.997+0000 7f5334445640 1 -- 172.21.15.132:0/4098352172 --> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f532c1d5620 con 0x7f531006d5b0 2024-03-13T08:11:17.997 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.997+0000 7f5322ffd640 1 -- 172.21.15.132:0/4098352172 <== mgr.4104 v2:172.21.15.132:6800/3847550149 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f531c037040 con 0x7f531006d5b0 2024-03-13T08:11:17.997 INFO:teuthology.orchestra.run.smithi132.stderr:instructed osd(s) 6 to deep-scrub 2024-03-13T08:11:18.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 10 librados: watch_flush enter 2024-03-13T08:11:18.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 10 librados: watch_flush exit 2024-03-13T08:11:18.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 20 client.19729.objecter shutdown clearing up homeless session... 2024-03-13T08:11:18.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 10 client.19729.objecter successfully canceled tick 2024-03-13T08:11:18.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 1 -- 172.21.15.132:0/4098352172 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f531006d5b0 msgr2=0x7f531006fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:18.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 1 --2- 172.21.15.132:0/4098352172 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f531006d5b0 0x7f531006fa70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f531c036000 tx=0x7f531c03d040 comp rx=0 tx=0).stop 2024-03-13T08:11:18.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 10 monclient: shutdown 2024-03-13T08:11:18.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:18.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 1 -- 172.21.15.132:0/4098352172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532c0af170 msgr2=0x7f532c1d7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:18.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 1 --2- 172.21.15.132:0/4098352172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532c0af170 0x7f532c1d7f20 secure :-1 s=READY pgs=4743 cs=0 l=1 rev1=1 crypto rx=0x7f532800b410 tx=0x7f532800b8d0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 1 -- 172.21.15.132:0/4098352172 shutdown_connections 2024-03-13T08:11:18.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 1 --2- 172.21.15.132:0/4098352172 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f531006d5b0 0x7f531006fa70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 1 --2- 172.21.15.132:0/4098352172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532c1434c0 0x7f532c1d8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 1 --2- 172.21.15.132:0/4098352172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532c0af170 0x7f532c1d7f20 unknown :-1 s=CLOSED pgs=4743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 1 --2- 172.21.15.132:0/4098352172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f532c06c800 0x7f532c124b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 1 -- 172.21.15.132:0/4098352172 >> 172.21.15.132:0/4098352172 conn(0x7f532c0b5d70 msgr2=0x7f532c0b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:18.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 1 -- 172.21.15.132:0/4098352172 shutdown_connections 2024-03-13T08:11:18.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 1 -- 172.21.15.132:0/4098352172 wait complete. 2024-03-13T08:11:18.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:17.999+0000 7f5334445640 1 librados: shutdown 2024-03-13T08:11:18.010 INFO:tasks.ceph:Scrubbing osd.7 2024-03-13T08:11:18.011 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-13T08:11:18.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.123+0000 7f0232206640 10 monclient: get_monmap_and_config 2024-03-13T08:11:18.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.123+0000 7f0232206640 10 monclient: build_initial_monmap 2024-03-13T08:11:18.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.124+0000 7f0232206640 10 monclient: monmap: 2024-03-13T08:11:18.125 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:18.125 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:18.125 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:18.124737+0000 2024-03-13T08:11:18.125 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:18.124737+0000 2024-03-13T08:11:18.125 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:18.126 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:18.126 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:18.126 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:18.126 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:18.126 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:18.126 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:18.126 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:18.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.124+0000 7f0232206640 1 Processor -- start 2024-03-13T08:11:18.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.125+0000 7f0232206640 1 -- start start 2024-03-13T08:11:18.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.125+0000 7f0232206640 10 monclient: init 2024-03-13T08:11:18.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.125+0000 7f0232206640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:18.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.125+0000 7f0232206640 10 monclient: _add_conns ranks=[3,2,1,4,0] 2024-03-13T08:11:18.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.125+0000 7f0232206640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f022c0b8760 0x7f022c1426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:18.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.125+0000 7f0232206640 10 monclient(hunting): picked mon.noname-a con 0x7f022c0b8760 addr v2:172.21.15.132:3300/0 2024-03-13T08:11:18.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.125+0000 7f0232206640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c0b8b40 0x7f022c0b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:18.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.125+0000 7f0232206640 10 monclient(hunting): picked mon.noname-b con 0x7f022c0b8b40 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:18.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.125+0000 7f0232206640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f022c0b24e0 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:18.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.125+0000 7f0232206640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022c0b1b00 con 0x7f022c0b24e0 2024-03-13T08:11:18.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.125+0000 7f0232206640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:18.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.125+0000 7f0232206640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f022c0b1400 con 0x7f022c0b8760 2024-03-13T08:11:18.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.125+0000 7f0232206640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:18.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.125+0000 7f0232206640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0b16e0 con 0x7f022c0b8b40 2024-03-13T08:11:18.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.125+0000 7f0232206640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:18.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.125+0000 7f0232206640 10 monclient(hunting): authenticate will time out at 3505.533769s 2024-03-13T08:11:18.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.126+0000 7f0223fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c0b8b40 0x7f022c0b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:18.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.126+0000 7f022b7fe640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f022c0b8760 0x7f022c1426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:18.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.126+0000 7f022bfff640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f022c0b24e0 0x7f022c0aea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.132:35302/0 (socket says 172.21.15.132:35302) 2024-03-13T08:11:18.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.126+0000 7f022bfff640 1 -- 172.21.15.132:0/253247718 learned_addr learned my addr 172.21.15.132:0/253247718 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T08:11:18.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.126+0000 7f022b7fe640 10 monclient(hunting): get_auth_request con 0x7f022c0b8760 auth_method 0 2024-03-13T08:11:18.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.126+0000 7f022b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:18.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.126+0000 7f022b7fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:18.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.126+0000 7f022b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:18.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.126+0000 7f0223fff640 10 monclient(hunting): get_auth_request con 0x7f022c0b8b40 auth_method 0 2024-03-13T08:11:18.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.126+0000 7f0223fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:18.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.126+0000 7f0223fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:18.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.126+0000 7f0223fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:18.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.127+0000 7f022b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:18.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.127+0000 7f022b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:18.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.127+0000 7f022b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:18.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.127+0000 7f0223fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:18.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.127+0000 7f0223fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:18.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.127+0000 7f0223fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:18.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.128+0000 7f022b7fe640 10 monclient(hunting): handle_auth_done global_id 16089 payload 274 2024-03-13T08:11:18.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.128+0000 7f022b7fe640 1 -- 172.21.15.132:0/253247718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c0b8b40 msgr2=0x7f022c0b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:18.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.128+0000 7f022affd640 1 -- 172.21.15.132:0/253247718 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775958688 0 0) 0x7f0224001490 con 0x7f022c0b24e0 2024-03-13T08:11:18.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.128+0000 7f022b7fe640 1 --2- 172.21.15.132:0/253247718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c0b8b40 0x7f022c0b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.128+0000 7f022b7fe640 1 -- 172.21.15.132:0/253247718 >> v1:172.21.15.195:6789/0 conn(0x7f022c0b24e0 legacy=0x7f022c0aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:18.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.129+0000 7f022b7fe640 10 monclient: _finish_hunting 0 2024-03-13T08:11:18.139 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.129+0000 7f022b7fe640 1 monclient: found mon.noname-a 2024-03-13T08:11:18.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.129+0000 7f022b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:18.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.129+0000 7f022b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T08:11:18.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.129+0000 7f022b7fe640 1 -- 172.21.15.132:0/253247718 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c0b4840 con 0x7f022c0b8760 2024-03-13T08:11:18.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.129+0000 7f022affd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-03-13T08:11:18.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.131+0000 7f022b7fe640 1 --2- 172.21.15.132:0/253247718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f022c0b8760 0x7f022c1426a0 secure :-1 s=READY pgs=3272 cs=0 l=1 rev1=1 crypto rx=0x7f021c0097a0 tx=0x7f021c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1b31b83e5046f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:18.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.133+0000 7f022affd640 1 -- 172.21.15.132:0/253247718 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c031780 con 0x7f022c0b8760 2024-03-13T08:11:18.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.133+0000 7f022affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:18.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.133+0000 7f022affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T08:11:18.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.133+0000 7f022affd640 10 monclient: dump: 2024-03-13T08:11:18.154 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:18.155 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:18.155 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.155 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.155 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:18.155 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:18.155 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:18.156 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:18.156 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:18.156 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:18.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.133+0000 7f022affd640 10 monclient: _finish_auth 0 2024-03-13T08:11:18.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.133+0000 7f022affd640 10 monclient: _check_auth_tickets 2024-03-13T08:11:18.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.133+0000 7f022affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:18.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.133+0000 7f0232206640 5 monclient: authenticate success, global_id 16089 2024-03-13T08:11:18.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.133+0000 7f0232206640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T08:11:18.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.135+0000 7f022affd640 1 -- 172.21.15.132:0/253247718 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021c037070 con 0x7f022c0b8760 2024-03-13T08:11:18.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.135+0000 7f022affd640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:18.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.135+0000 7f0232206640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:18.157 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.135+0000 7f022affd640 1 -- 172.21.15.132:0/253247718 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c031b40 con 0x7f022c0b8760 2024-03-13T08:11:18.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.135+0000 7f022affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:18.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.135+0000 7f0232206640 4 set_mon_vals no callback set 2024-03-13T08:11:18.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.135+0000 7f022affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T08:11:18.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.135+0000 7f022affd640 10 monclient: dump: 2024-03-13T08:11:18.159 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:18.159 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:18.159 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.159 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.159 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:18.159 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:18.159 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:18.160 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:18.160 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:18.160 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:18.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.135+0000 7f0232206640 10 monclient: shutdown 2024-03-13T08:11:18.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.135+0000 7f0232206640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:18.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.135+0000 7f0232206640 1 -- 172.21.15.132:0/253247718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f022c0b8760 msgr2=0x7f022c1426a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:18.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.135+0000 7f0232206640 1 --2- 172.21.15.132:0/253247718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f022c0b8760 0x7f022c1426a0 secure :-1 s=READY pgs=3272 cs=0 l=1 rev1=1 crypto rx=0x7f021c0097a0 tx=0x7f021c02b820 comp rx=0 tx=0).stop 2024-03-13T08:11:18.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.136+0000 7f0232206640 1 -- 172.21.15.132:0/253247718 shutdown_connections 2024-03-13T08:11:18.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.136+0000 7f0232206640 1 --2- 172.21.15.132:0/253247718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c0b8b40 0x7f022c0b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.136+0000 7f0232206640 1 --2- 172.21.15.132:0/253247718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f022c0b8760 0x7f022c1426a0 unknown :-1 s=CLOSED pgs=3272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.136+0000 7f0232206640 1 -- 172.21.15.132:0/253247718 >> 172.21.15.132:0/253247718 conn(0x7f022c06d4f0 msgr2=0x7f022c0b8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:18.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.136+0000 7f0232206640 1 -- 172.21.15.132:0/253247718 shutdown_connections 2024-03-13T08:11:18.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.136+0000 7f0232206640 1 -- 172.21.15.132:0/253247718 wait complete. 2024-03-13T08:11:18.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.136+0000 7f0232206640 10 monclient: build_initial_monmap 2024-03-13T08:11:18.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.136+0000 7f0232206640 10 monclient: monmap: 2024-03-13T08:11:18.162 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:18.162 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:18.163 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:18.163 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:18.163 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:18.163 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:18.163 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:18.163 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:18.163 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:18.163 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:18.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 1 librados: starting msgr at 2024-03-13T08:11:18.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 1 librados: starting objecter 2024-03-13T08:11:18.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 1 Processor -- start 2024-03-13T08:11:18.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 1 -- start start 2024-03-13T08:11:18.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 1 librados: setting wanted keys 2024-03-13T08:11:18.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 1 librados: calling monclient init 2024-03-13T08:11:18.164 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 10 monclient: init 2024-03-13T08:11:18.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:18.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 10 monclient: _add_conns ranks=[1,0,2] 2024-03-13T08:11:18.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f022c0b8760 0x7f022c14c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:18.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 10 monclient(hunting): picked mon.noname-b con 0x7f022c0b8760 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:18.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f022c1db730 0x7f022c0ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:18.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 10 monclient(hunting): picked mon.noname-a con 0x7f022c1db730 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:18.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c0ab6d0 0x7f022c0abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:18.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 10 monclient(hunting): picked mon.noname-c con 0x7f022c0ab6d0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:18.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:18.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0b00c0 con 0x7f022c1db730 2024-03-13T08:11:18.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:18.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0aff40 con 0x7f022c0b8760 2024-03-13T08:11:18.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:18.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0b0240 con 0x7f022c0ab6d0 2024-03-13T08:11:18.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:18.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.137+0000 7f0232206640 10 monclient(hunting): authenticate will time out at 3505.545892s 2024-03-13T08:11:18.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f0223fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f022c1db730 0x7f022c0ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:18.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f022b7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f022c0b8760 0x7f022c14c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:18.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f0223fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f022c1db730 0x7f022c0ae150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56238/0 (socket says 172.21.15.132:56238) 2024-03-13T08:11:18.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f0223fff640 1 -- 172.21.15.132:0/4243583178 learned_addr learned my addr 172.21.15.132:0/4243583178 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:18.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f022bfff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c0ab6d0 0x7f022c0abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:18.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f0223fff640 10 monclient(hunting): get_auth_request con 0x7f022c1db730 auth_method 0 2024-03-13T08:11:18.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f0223fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:18.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f0223fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:18.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f0223fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:18.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f022bfff640 10 monclient(hunting): get_auth_request con 0x7f022c0ab6d0 auth_method 0 2024-03-13T08:11:18.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f022bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:18.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f022bfff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:18.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f022bfff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:18.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f022b7fe640 10 monclient(hunting): get_auth_request con 0x7f022c0b8760 auth_method 0 2024-03-13T08:11:18.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f022b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:18.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f022b7fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:18.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.138+0000 7f022b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:18.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.139+0000 7f0223fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:18.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.139+0000 7f0223fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:18.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.139+0000 7f0223fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:18.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.139+0000 7f022b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:18.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.139+0000 7f022b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:18.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.139+0000 7f022b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:18.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.139+0000 7f022bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:18.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.139+0000 7f022bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:18.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.139+0000 7f022bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:18.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.142+0000 7f0223fff640 10 monclient(hunting): handle_auth_done global_id 16092 payload 824 2024-03-13T08:11:18.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.142+0000 7f0223fff640 1 -- 172.21.15.132:0/4243583178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c0ab6d0 msgr2=0x7f022c0abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:18.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.142+0000 7f0223fff640 1 --2- 172.21.15.132:0/4243583178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c0ab6d0 0x7f022c0abb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.142+0000 7f0223fff640 1 -- 172.21.15.132:0/4243583178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f022c0b8760 msgr2=0x7f022c14c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:18.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.142+0000 7f0223fff640 1 --2- 172.21.15.132:0/4243583178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f022c0b8760 0x7f022c14c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.142+0000 7f0223fff640 10 monclient: _finish_hunting 0 2024-03-13T08:11:18.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.142+0000 7f0223fff640 1 monclient: found mon.noname-a 2024-03-13T08:11:18.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.142+0000 7f0223fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:18.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.142+0000 7f0223fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T08:11:18.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.142+0000 7f0223fff640 1 -- 172.21.15.132:0/4243583178 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c1dbd30 con 0x7f022c1db730 2024-03-13T08:11:18.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.144+0000 7f0223fff640 1 --2- 172.21.15.132:0/4243583178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f022c1db730 0x7f022c0ae150 secure :-1 s=READY pgs=3273 cs=0 l=1 rev1=1 crypto rx=0x7f021400e870 tx=0x7f021400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:18.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.144+0000 7f0228ff9640 10 client.?.objecter ms_handle_connect 0x7f022c1db730 2024-03-13T08:11:18.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.144+0000 7f0228ff9640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:18.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0228ff9640 1 -- 172.21.15.132:0/4243583178 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214017070 con 0x7f022c1db730 2024-03-13T08:11:18.171 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0228ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:18.171 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0228ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T08:11:18.171 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0228ff9640 10 monclient: dump: 2024-03-13T08:11:18.171 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:18.171 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:18.171 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.171 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.172 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:18.172 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:18.172 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:18.172 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:18.172 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:18.172 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:18.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0228ff9640 10 monclient: _finish_auth 0 2024-03-13T08:11:18.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0228ff9640 10 monclient: _check_auth_tickets 2024-03-13T08:11:18.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0228ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:18.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0228ff9640 1 -- 172.21.15.132:0/4243583178 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021400c8e0 con 0x7f022c1db730 2024-03-13T08:11:18.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0232206640 5 monclient: authenticate success, global_id 16092 2024-03-13T08:11:18.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0232206640 10 monclient: _renew_subs 2024-03-13T08:11:18.174 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0232206640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T08:11:18.174 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0232206640 1 -- 172.21.15.132:0/4243583178 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022c0ae690 con 0x7f022c1db730 2024-03-13T08:11:18.174 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0228ff9640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:18.174 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0228ff9640 1 -- 172.21.15.132:0/4243583178 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021400ccd0 con 0x7f022c1db730 2024-03-13T08:11:18.174 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0228ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:18.175 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0228ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T08:11:18.175 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0228ff9640 10 monclient: dump: 2024-03-13T08:11:18.175 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:18.175 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:18.175 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.175 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.176 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:18.176 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:18.176 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:18.176 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:18.176 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:18.176 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:18.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f022a7fc640 4 set_mon_vals no callback set 2024-03-13T08:11:18.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0232206640 10 client.16092.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:18.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0232206640 10 monclient: _renew_subs 2024-03-13T08:11:18.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0232206640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T08:11:18.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0232206640 1 -- 172.21.15.132:0/4243583178 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022c0aa8c0 con 0x7f022c1db730 2024-03-13T08:11:18.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.148+0000 7f0232206640 1 librados: init done 2024-03-13T08:11:18.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.149+0000 7f0232206640 10 client.16092.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:18.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.149+0000 7f0232206640 15 client.16092.objecter _session_command_op_assign -1 1 2024-03-13T08:11:18.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.149+0000 7f0232206640 20 client.16092.objecter _get_session osd=-1 returning homeless 2024-03-13T08:11:18.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.149+0000 7f0232206640 10 client.16092.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:18.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.149+0000 7f0232206640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T08:11:18.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.149+0000 7f0232206640 1 -- 172.21.15.132:0/4243583178 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f022c04f000 con 0x7f022c1db730 2024-03-13T08:11:18.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.167+0000 7f0228ff9640 1 -- 172.21.15.132:0/4243583178 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f021401a070 con 0x7f022c1db730 2024-03-13T08:11:18.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.168+0000 7f0228ff9640 1 --2- 172.21.15.132:0/4243583178 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f020c06d560 0x7f020c06fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:18.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.168+0000 7f022b7fe640 1 --2- 172.21.15.132:0/4243583178 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f020c06d560 0x7f020c06fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:18.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.168+0000 7f022b7fe640 10 monclient: get_auth_request con 0x7f020c06d560 auth_method 0 2024-03-13T08:11:18.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.168+0000 7f0228ff9640 10 client.16092.objecter ms_handle_connect 0x7f020c06d560 2024-03-13T08:11:18.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.168+0000 7f022b7fe640 1 --2- 172.21.15.132:0/4243583178 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f020c06d560 0x7f020c06fa20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f021c036000 tx=0x7f021c03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:18.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.170+0000 7f0228ff9640 1 -- 172.21.15.132:0/4243583178 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f0214057190 con 0x7f022c1db730 2024-03-13T08:11:18.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.170+0000 7f0228ff9640 10 client.16092.objecter ms_dispatch 0x7f022c000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:18.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.170+0000 7f0228ff9640 3 client.16092.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:18.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.170+0000 7f0228ff9640 3 client.16092.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:18.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.170+0000 7f0228ff9640 10 client.16092.objecter checking command 1 2024-03-13T08:11:18.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.170+0000 7f0228ff9640 1 --2- 172.21.15.132:0/4243583178 >> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] conn(0x7f020c0751d0 0x7f020c077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:18.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.170+0000 7f0228ff9640 20 client.16092.objecter _get_session s=0x7f020c074b00 osd=7 3 2024-03-13T08:11:18.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.170+0000 7f0228ff9640 20 client.16092.objecter put_session s=0x7f020c074b00 osd=7 3 2024-03-13T08:11:18.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.170+0000 7f0228ff9640 15 client.16092.objecter _session_command_op_remove -1 1 2024-03-13T08:11:18.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.170+0000 7f0228ff9640 20 client.16092.objecter _get_session s=0x7f020c074b00 osd=7 3 2024-03-13T08:11:18.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.170+0000 7f0228ff9640 20 client.16092.objecter get_session s=0x7f020c074b00 osd=7 3 2024-03-13T08:11:18.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.170+0000 7f0228ff9640 15 client.16092.objecter _session_command_op_assign 7 1 2024-03-13T08:11:18.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.170+0000 7f0228ff9640 20 client.16092.objecter put_session s=0x7f020c074b00 osd=7 4 2024-03-13T08:11:18.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.170+0000 7f0228ff9640 10 client.16092.objecter _send_command 1 2024-03-13T08:11:18.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.170+0000 7f0228ff9640 1 -- 172.21.15.132:0/4243583178 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f021400be50 con 0x7f020c0751d0 2024-03-13T08:11:18.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.170+0000 7f0228ff9640 20 client.16092.objecter dump_active .. 0 homeless 2024-03-13T08:11:18.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.171+0000 7f022bfff640 1 --2- 172.21.15.132:0/4243583178 >> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] conn(0x7f020c0751d0 0x7f020c077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:18.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.171+0000 7f022bfff640 10 monclient: get_auth_request con 0x7f020c0751d0 auth_method 0 2024-03-13T08:11:18.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.172+0000 7f0228ff9640 1 -- 172.21.15.132:0/4243583178 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=1293) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0214055390 con 0x7f022c1db730 2024-03-13T08:11:18.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.172+0000 7f0228ff9640 10 monclient: handle_get_version_reply finishing 1 version 1293 2024-03-13T08:11:18.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.174+0000 7f022bfff640 1 --2- 172.21.15.132:0/4243583178 >> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] conn(0x7f020c0751d0 0x7f020c077670 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:18.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.174+0000 7f0228ff9640 10 client.16092.objecter ms_handle_connect 0x7f020c0751d0 2024-03-13T08:11:18.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.198+0000 7f0228ff9640 1 -- 172.21.15.132:0/4243583178 <== osd.7 v2:172.21.15.195:6820/3378300314 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f021400be50 con 0x7f020c0751d0 2024-03-13T08:11:18.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.198+0000 7f0228ff9640 10 client.16092.objecter ms_dispatch 0x7f022c000b90 command_reply(tid 1: 0 ) v1 2024-03-13T08:11:18.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.198+0000 7f0228ff9640 10 client.16092.objecter _finish_command 1 = system:0 2024-03-13T08:11:18.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.198+0000 7f0228ff9640 20 client.16092.objecter put_session s=0x7f020c074b00 osd=7 4 2024-03-13T08:11:18.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.198+0000 7f0228ff9640 15 client.16092.objecter _session_command_op_remove 7 1 2024-03-13T08:11:18.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.213+0000 7f0232206640 10 client.16092.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-03-13T08:11:18.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.213+0000 7f0232206640 15 client.16092.objecter _session_command_op_assign -1 2 2024-03-13T08:11:18.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.213+0000 7f0232206640 20 client.16092.objecter _get_session s=0x7f020c074b00 osd=7 3 2024-03-13T08:11:18.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.213+0000 7f0232206640 20 client.16092.objecter put_session s=0x7f020c074b00 osd=7 3 2024-03-13T08:11:18.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.213+0000 7f0232206640 20 client.16092.objecter _get_session s=0x7f020c074b00 osd=7 3 2024-03-13T08:11:18.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.213+0000 7f0232206640 15 client.16092.objecter _session_command_op_remove -1 2 2024-03-13T08:11:18.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.213+0000 7f0232206640 20 client.16092.objecter get_session s=0x7f020c074b00 osd=7 3 2024-03-13T08:11:18.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.213+0000 7f0232206640 15 client.16092.objecter _session_command_op_assign 7 2 2024-03-13T08:11:18.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.213+0000 7f0232206640 20 client.16092.objecter put_session s=0x7f020c074b00 osd=7 4 2024-03-13T08:11:18.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.213+0000 7f0232206640 10 client.16092.objecter _send_command 2 2024-03-13T08:11:18.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.213+0000 7f0232206640 1 -- 172.21.15.132:0/4243583178 --> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f022c14cd30 con 0x7f020c0751d0 2024-03-13T08:11:18.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0228ff9640 1 -- 172.21.15.132:0/4243583178 <== osd.7 v2:172.21.15.195:6820/3378300314 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f022c14cd30 con 0x7f020c0751d0 2024-03-13T08:11:18.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0228ff9640 10 client.16092.objecter ms_dispatch 0x7f022c000b90 command_reply(tid 2: 0 ) v1 2024-03-13T08:11:18.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0228ff9640 10 client.16092.objecter _finish_command 2 = system:0 2024-03-13T08:11:18.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0228ff9640 20 client.16092.objecter put_session s=0x7f020c074b00 osd=7 4 2024-03-13T08:11:18.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0228ff9640 15 client.16092.objecter _session_command_op_remove 7 2 2024-03-13T08:11:18.227 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-13T08:11:18.227 INFO:teuthology.orchestra.run.smithi132.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-13T08:11:18.227 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-13T08:11:18.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 10 librados: watch_flush enter 2024-03-13T08:11:18.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 10 librados: watch_flush exit 2024-03-13T08:11:18.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 10 client.16092.objecter close_session for osd.7 2024-03-13T08:11:18.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 1 -- 172.21.15.132:0/4243583178 >> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] conn(0x7f020c0751d0 msgr2=0x7f020c077670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:18.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 1 --2- 172.21.15.132:0/4243583178 >> [v2:172.21.15.195:6820/3378300314,v1:172.21.15.195:6823/3378300314] conn(0x7f020c0751d0 0x7f020c077670 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 20 client.16092.objecter put_session s=0x7f020c074b00 osd=7 1 2024-03-13T08:11:18.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 20 client.16092.objecter shutdown clearing up homeless session... 2024-03-13T08:11:18.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 10 client.16092.objecter successfully canceled tick 2024-03-13T08:11:18.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 1 -- 172.21.15.132:0/4243583178 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f020c06d560 msgr2=0x7f020c06fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:18.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 1 --2- 172.21.15.132:0/4243583178 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f020c06d560 0x7f020c06fa20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f021c036000 tx=0x7f021c03d040 comp rx=0 tx=0).stop 2024-03-13T08:11:18.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 10 monclient: shutdown 2024-03-13T08:11:18.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:18.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 1 -- 172.21.15.132:0/4243583178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f022c1db730 msgr2=0x7f022c0ae150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:18.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 1 --2- 172.21.15.132:0/4243583178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f022c1db730 0x7f022c0ae150 secure :-1 s=READY pgs=3273 cs=0 l=1 rev1=1 crypto rx=0x7f021400e870 tx=0x7f021400ed30 comp rx=0 tx=0).stop 2024-03-13T08:11:18.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f022bfff640 1 -- 172.21.15.132:0/4243583178 reap_dead start 2024-03-13T08:11:18.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 1 -- 172.21.15.132:0/4243583178 shutdown_connections 2024-03-13T08:11:18.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 1 -- 172.21.15.132:0/4243583178 >> 172.21.15.132:0/4243583178 conn(0x7f022c06d4f0 msgr2=0x7f022c0b6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:18.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 1 -- 172.21.15.132:0/4243583178 shutdown_connections 2024-03-13T08:11:18.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.226+0000 7f0232206640 1 -- 172.21.15.132:0/4243583178 wait complete. 2024-03-13T08:11:18.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.227+0000 7f0232206640 1 librados: shutdown 2024-03-13T08:11:18.238 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-13T08:11:18.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.352+0000 7f05d101d640 10 monclient: get_monmap_and_config 2024-03-13T08:11:18.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.352+0000 7f05d101d640 10 monclient: build_initial_monmap 2024-03-13T08:11:18.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.352+0000 7f05d101d640 10 monclient: monmap: 2024-03-13T08:11:18.353 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:18.353 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:18.353 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:18.353204+0000 2024-03-13T08:11:18.353 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:18.353204+0000 2024-03-13T08:11:18.353 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:18.353 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:18.354 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:18.354 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:18.354 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:18.354 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:18.355 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:18.355 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:18.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 1 Processor -- start 2024-03-13T08:11:18.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 1 -- start start 2024-03-13T08:11:18.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 10 monclient: init 2024-03-13T08:11:18.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:18.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 10 monclient: _add_conns ranks=[2,4,0,3,1] 2024-03-13T08:11:18.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05cc0b5320 0x7f05cc0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:18.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 10 monclient(hunting): picked mon.noname-b con 0x7f05cc0b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:18.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f05cc0b1ac0 0x7f05cc0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:18.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 10 monclient(hunting): picked mon.noname-c con 0x7f05cc0b1ac0 addr v2:172.21.15.195:3300/0 2024-03-13T08:11:18.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f05cc13f630 addr v1:172.21.15.132:6789/0 2024-03-13T08:11:18.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05cc0b1150 con 0x7f05cc13f630 2024-03-13T08:11:18.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:18.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05cc0b5700 con 0x7f05cc0b1ac0 2024-03-13T08:11:18.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:18.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc0b13c0 con 0x7f05cc0b5320 2024-03-13T08:11:18.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:18.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.353+0000 7f05d101d640 10 monclient(hunting): authenticate will time out at 3505.761778s 2024-03-13T08:11:18.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.384+0000 7f05cb59e640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f05cc13f630 0x7f05cc13fa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:50460/0 (socket says 172.21.15.132:50460) 2024-03-13T08:11:18.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.384+0000 7f05cb59e640 1 -- 172.21.15.132:0/1092894505 learned_addr learned my addr 172.21.15.132:0/1092894505 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T08:11:18.391 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.388+0000 7f05c9d9b640 1 -- 172.21.15.132:0/1092894505 <== mon.0 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809816093 0 0) 0x7f05cc0b1150 con 0x7f05cc13f630 2024-03-13T08:11:18.391 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.388+0000 7f05c9d9b640 10 monclient(hunting): _negotiate 2024-03-13T08:11:18.391 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.388+0000 7f05c9d9b640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:18.391 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.388+0000 7f05c9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:18.391 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.388+0000 7f05c9d9b640 10 monclient(hunting): my global_id is 16095 2024-03-13T08:11:18.391 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.388+0000 7f05c9d9b640 1 -- 172.21.15.132:0/1092894505 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05b4003520 con 0x7f05cc13f630 2024-03-13T08:11:18.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 1 -- 172.21.15.132:0/1092894505 <== mon.0 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1215152956 0 0) 0x7f05b4003520 con 0x7f05cc13f630 2024-03-13T08:11:18.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 1 -- 172.21.15.132:0/1092894505 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05cc0b5320 msgr2=0x7f05cc0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:18.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 1 --2- 172.21.15.132:0/1092894505 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05cc0b5320 0x7f05cc0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 1 -- 172.21.15.132:0/1092894505 >> v2:172.21.15.195:3300/0 conn(0x7f05cc0b1ac0 msgr2=0x7f05cc0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:18.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 1 --2- 172.21.15.132:0/1092894505 >> v2:172.21.15.195:3300/0 conn(0x7f05cc0b1ac0 0x7f05cc0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient: _finish_hunting 0 2024-03-13T08:11:18.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 1 monclient: found mon.noname-a-legacy 2024-03-13T08:11:18.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:18.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.132:6789/0 2024-03-13T08:11:18.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 1 -- 172.21.15.132:0/1092894505 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc0b16a0 con 0x7f05cc13f630 2024-03-13T08:11:18.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient: _finish_auth 0 2024-03-13T08:11:18.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient: _check_auth_tickets 2024-03-13T08:11:18.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:18.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 1 -- 172.21.15.132:0/1092894505 <== mon.0 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1639563336 0 0) 0x7f05c40019f0 con 0x7f05cc13f630 2024-03-13T08:11:18.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05d101d640 5 monclient: authenticate success, global_id 16095 2024-03-13T08:11:18.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05d101d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T08:11:18.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:18.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-03-13T08:11:18.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient: dump: 2024-03-13T08:11:18.401 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:18.401 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:18.401 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.401 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.401 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:18.402 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:18.402 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:18.402 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:18.402 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:18.402 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:18.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] but i'm connected to v1:172.21.15.132:6789/0, reconnecting 2024-03-13T08:11:18.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:18.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 1 -- 172.21.15.132:0/1092894505 >> v1:172.21.15.132:6789/0 conn(0x7f05cc13f630 legacy=0x7f05cc13fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:18.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient: _add_conns ranks=[0,2,1] 2024-03-13T08:11:18.403 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 1 --2- 172.21.15.132:0/1092894505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05b40039b0 0x7f05b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:18.403 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient(hunting): picked mon.a con 0x7f05b40039b0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:18.403 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 1 --2- 172.21.15.132:0/1092894505 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05cc0b5320 0x7f05b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:18.403 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient(hunting): picked mon.c con 0x7f05cc0b5320 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:18.403 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 1 --2- 172.21.15.132:0/1092894505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05b4004f40 0x7f05b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:18.403 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient(hunting): picked mon.b con 0x7f05b4004f40 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:18.403 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:18.403 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 1 -- 172.21.15.132:0/1092894505 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc0b5700 con 0x7f05b40039b0 2024-03-13T08:11:18.403 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:18.403 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 1 -- 172.21.15.132:0/1092894505 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc0b13c0 con 0x7f05b4004f40 2024-03-13T08:11:18.404 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:18.404 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 1 -- 172.21.15.132:0/1092894505 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b400bf00 con 0x7f05cc0b5320 2024-03-13T08:11:18.404 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:18.404 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05c9d9b640 10 monclient(hunting): _finish_auth 0 2024-03-13T08:11:18.404 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.392+0000 7f05d101d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-03-13T08:11:18.404 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.396+0000 7f05cad9d640 1 --2- 172.21.15.132:0/1092894505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05b40039b0 0x7f05b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:18.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.398+0000 7f05ca59c640 1 --2- 172.21.15.132:0/1092894505 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05cc0b5320 0x7f05b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.398+0000 7f05cad9d640 10 monclient(hunting): get_auth_request con 0x7f05b40039b0 auth_method 0 2024-03-13T08:11:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.398+0000 7f05cad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.398+0000 7f05cad9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.398+0000 7f05cad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-03-13T08:11:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.398+0000 7f05ca59c640 10 monclient(hunting): get_auth_request con 0x7f05cc0b5320 auth_method 0 2024-03-13T08:11:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.398+0000 7f05ca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.398+0000 7f05ca59c640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.398+0000 7f05ca59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-03-13T08:11:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.398+0000 7f05cad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.398+0000 7f05cad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.398+0000 7f05cad9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-03-13T08:11:18.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.398+0000 7f05ca59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.398+0000 7f05ca59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.398+0000 7f05ca59c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-03-13T08:11:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.400+0000 7f05cad9d640 10 monclient(hunting): handle_auth_done global_id 16095 payload 293 2024-03-13T08:11:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.400+0000 7f05cad9d640 1 -- 172.21.15.132:0/1092894505 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05cc0b5320 msgr2=0x7f05b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.400+0000 7f05cad9d640 1 --2- 172.21.15.132:0/1092894505 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05cc0b5320 0x7f05b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.400+0000 7f05cad9d640 1 -- 172.21.15.132:0/1092894505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05b4004f40 msgr2=0x7f05b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.400+0000 7f05cad9d640 1 --2- 172.21.15.132:0/1092894505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05b4004f40 0x7f05b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.400+0000 7f05cad9d640 10 monclient: _finish_hunting 0 2024-03-13T08:11:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.400+0000 7f05cad9d640 1 monclient: found mon.a 2024-03-13T08:11:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.400+0000 7f05cad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.400+0000 7f05cad9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T08:11:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.400+0000 7f05cad9d640 1 -- 172.21.15.132:0/1092894505 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b4003520 con 0x7f05b40039b0 2024-03-13T08:11:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.400+0000 7f05cad9d640 10 monclient: _finish_auth 0 2024-03-13T08:11:18.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.400+0000 7f05cad9d640 10 monclient: _check_auth_tickets 2024-03-13T08:11:18.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.400+0000 7f05cad9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:18.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.402+0000 7f05cad9d640 1 --2- 172.21.15.132:0/1092894505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05b40039b0 0x7f05b4003dd0 secure :-1 s=READY pgs=3275 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009860 tx=0x7f05b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5471f1a6b799ac7e server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:18.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.404+0000 7f05c9d9b640 1 -- 172.21.15.132:0/1092894505 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc037070 con 0x7f05b40039b0 2024-03-13T08:11:18.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.404+0000 7f05c9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:18.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.404+0000 7f05c9d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T08:11:18.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.404+0000 7f05c9d9b640 10 monclient: dump: 2024-03-13T08:11:18.423 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:18.423 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:18.423 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.423 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.423 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:18.423 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:18.423 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:18.423 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:18.424 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:18.424 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:18.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.404+0000 7f05d101d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T08:11:18.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.406+0000 7f05c9d9b640 1 -- 172.21.15.132:0/1092894505 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05bc032620 con 0x7f05b40039b0 2024-03-13T08:11:18.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.406+0000 7f05c9d9b640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:18.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.406+0000 7f05c9d9b640 1 -- 172.21.15.132:0/1092894505 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc009460 con 0x7f05b40039b0 2024-03-13T08:11:18.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.406+0000 7f05c9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:18.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.406+0000 7f05c9d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T08:11:18.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.406+0000 7f05c9d9b640 10 monclient: dump: 2024-03-13T08:11:18.425 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:18.425 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:18.425 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.425 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.425 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:18.425 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:18.425 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:18.425 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:18.425 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:18.425 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:18.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.406+0000 7f05d101d640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:18.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.406+0000 7f05d101d640 4 set_mon_vals no callback set 2024-03-13T08:11:18.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.407+0000 7f05d101d640 10 monclient: shutdown 2024-03-13T08:11:18.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.407+0000 7f05d101d640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:18.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.407+0000 7f05d101d640 1 -- 172.21.15.132:0/1092894505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05b40039b0 msgr2=0x7f05b4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:18.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.407+0000 7f05d101d640 1 --2- 172.21.15.132:0/1092894505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05b40039b0 0x7f05b4003dd0 secure :-1 s=READY pgs=3275 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009860 tx=0x7f05b4005830 comp rx=0 tx=0).stop 2024-03-13T08:11:18.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.407+0000 7f05cb59e640 1 -- 172.21.15.132:0/1092894505 reap_dead start 2024-03-13T08:11:18.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.407+0000 7f05d101d640 1 -- 172.21.15.132:0/1092894505 shutdown_connections 2024-03-13T08:11:18.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.407+0000 7f05d101d640 1 -- 172.21.15.132:0/1092894505 >> 172.21.15.132:0/1092894505 conn(0x7f05cc06d2b0 msgr2=0x7f05cc06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:18.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.407+0000 7f05d101d640 1 -- 172.21.15.132:0/1092894505 shutdown_connections 2024-03-13T08:11:18.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.407+0000 7f05d101d640 1 -- 172.21.15.132:0/1092894505 wait complete. 2024-03-13T08:11:18.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.408+0000 7f05d101d640 10 monclient: build_initial_monmap 2024-03-13T08:11:18.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.408+0000 7f05d101d640 10 monclient: monmap: 2024-03-13T08:11:18.426 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:18.426 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:18.426 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:18.426 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:18.426 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:18.426 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:18.427 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:18.427 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:18.427 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:18.427 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:18.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.409+0000 7f05d101d640 1 librados: starting msgr at 2024-03-13T08:11:18.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.409+0000 7f05d101d640 1 librados: starting objecter 2024-03-13T08:11:18.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.409+0000 7f05d101d640 1 Processor -- start 2024-03-13T08:11:18.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.409+0000 7f05d101d640 1 -- start start 2024-03-13T08:11:18.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.409+0000 7f05d101d640 1 librados: setting wanted keys 2024-03-13T08:11:18.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.409+0000 7f05d101d640 1 librados: calling monclient init 2024-03-13T08:11:18.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.409+0000 7f05d101d640 10 monclient: init 2024-03-13T08:11:18.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.410+0000 7f05d101d640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:18.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.410+0000 7f05d101d640 10 monclient: _add_conns ranks=[1,0,2] 2024-03-13T08:11:18.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.410+0000 7f05d101d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05b40039b0 0x7f05cc0ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:18.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.410+0000 7f05d101d640 10 monclient(hunting): picked mon.noname-b con 0x7f05b40039b0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:18.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.410+0000 7f05d101d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05cc140be0 0x7f05cc1dafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:18.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.410+0000 7f05d101d640 10 monclient(hunting): picked mon.noname-a con 0x7f05cc140be0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:18.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.410+0000 7f05d101d640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05cc141060 0x7f05cc1db4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:18.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.410+0000 7f05d101d640 10 monclient(hunting): picked mon.noname-c con 0x7f05cc141060 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:18.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.410+0000 7f05d101d640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:18.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.410+0000 7f05d101d640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc0b4800 con 0x7f05cc140be0 2024-03-13T08:11:18.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.410+0000 7f05d101d640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:18.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.410+0000 7f05d101d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc0afd40 con 0x7f05b40039b0 2024-03-13T08:11:18.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.410+0000 7f05d101d640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:18.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.410+0000 7f05d101d640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc0b4980 con 0x7f05cc141060 2024-03-13T08:11:18.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.410+0000 7f05d101d640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:18.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.410+0000 7f05d101d640 10 monclient(hunting): authenticate will time out at 3505.818880s 2024-03-13T08:11:18.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.411+0000 7f05cb59e640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05cc141060 0x7f05cc1db4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:18.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.411+0000 7f05cb59e640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05cc141060 0x7f05cc1db4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:59926/0 (socket says 172.21.15.132:59926) 2024-03-13T08:11:18.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.411+0000 7f05cb59e640 1 -- 172.21.15.132:0/2237837776 learned_addr learned my addr 172.21.15.132:0/2237837776 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:18.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.411+0000 7f05cb59e640 10 monclient(hunting): get_auth_request con 0x7f05cc141060 auth_method 0 2024-03-13T08:11:18.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.411+0000 7f05cb59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:18.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.411+0000 7f05cb59e640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:18.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.411+0000 7f05cb59e640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:18.429 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.412+0000 7f05cb59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:18.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.412+0000 7f05cb59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:18.430 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.412+0000 7f05cb59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:18.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.412+0000 7f05cad9d640 1 --2- 172.21.15.132:0/2237837776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05b40039b0 0x7f05cc0ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:18.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.413+0000 7f05cad9d640 10 monclient(hunting): get_auth_request con 0x7f05b40039b0 auth_method 0 2024-03-13T08:11:18.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.413+0000 7f05cad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:18.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.413+0000 7f05cad9d640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:18.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.413+0000 7f05cad9d640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:18.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.414+0000 7f05cad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:18.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.414+0000 7f05cad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:18.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.414+0000 7f05cad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:18.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.415+0000 7f05cb59e640 10 monclient(hunting): handle_auth_done global_id 19040 payload 824 2024-03-13T08:11:18.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.415+0000 7f05cb59e640 1 -- 172.21.15.132:0/2237837776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05b40039b0 msgr2=0x7f05cc0ab6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:18.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.415+0000 7f05cb59e640 1 --2- 172.21.15.132:0/2237837776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05b40039b0 0x7f05cc0ab6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.415+0000 7f05cb59e640 1 -- 172.21.15.132:0/2237837776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05cc140be0 msgr2=0x7f05cc1dafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:18.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.415+0000 7f05cb59e640 1 --2- 172.21.15.132:0/2237837776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05cc140be0 0x7f05cc1dafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.415+0000 7f05cb59e640 10 monclient: _finish_hunting 0 2024-03-13T08:11:18.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.415+0000 7f05cb59e640 1 monclient: found mon.noname-c 2024-03-13T08:11:18.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.415+0000 7f05cb59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:18.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.415+0000 7f05cb59e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T08:11:18.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.415+0000 7f05cb59e640 1 -- 172.21.15.132:0/2237837776 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc1403e0 con 0x7f05cc141060 2024-03-13T08:11:18.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.418+0000 7f05cb59e640 1 --2- 172.21.15.132:0/2237837776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05cc141060 0x7f05cc1db4f0 secure :-1 s=READY pgs=3545 cs=0 l=1 rev1=1 crypto rx=0x7f05cc13fa10 tx=0x7f05c400cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:18.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.418+0000 7f05b37fe640 10 client.?.objecter ms_handle_connect 0x7f05cc141060 2024-03-13T08:11:18.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.418+0000 7f05b37fe640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:18.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05b37fe640 1 -- 172.21.15.132:0/2237837776 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c4004d60 con 0x7f05cc141060 2024-03-13T08:11:18.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05b37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:18.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05b37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T08:11:18.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05b37fe640 10 monclient: dump: 2024-03-13T08:11:18.436 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:18.436 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:18.437 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.437 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.437 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:18.437 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:18.437 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:18.437 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:18.437 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:18.437 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:18.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05b37fe640 10 monclient: _finish_auth 0 2024-03-13T08:11:18.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05b37fe640 10 monclient: _check_auth_tickets 2024-03-13T08:11:18.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05b37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:18.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05b37fe640 1 -- 172.21.15.132:0/2237837776 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c400a5b0 con 0x7f05cc141060 2024-03-13T08:11:18.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05b37fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:18.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05b37fe640 1 -- 172.21.15.132:0/2237837776 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c400a9a0 con 0x7f05cc141060 2024-03-13T08:11:18.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05b37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:18.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05b37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T08:11:18.439 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05b37fe640 10 monclient: dump: 2024-03-13T08:11:18.439 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:18.439 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:18.439 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.439 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:18.439 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:18.439 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:18.439 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:18.439 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:18.439 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:18.439 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:18.439 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05d101d640 5 monclient: authenticate success, global_id 19040 2024-03-13T08:11:18.439 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05d101d640 10 monclient: _renew_subs 2024-03-13T08:11:18.439 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05d101d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:18.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05d101d640 1 -- 172.21.15.132:0/2237837776 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05cc1dba30 con 0x7f05cc141060 2024-03-13T08:11:18.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05c959a640 4 set_mon_vals no callback set 2024-03-13T08:11:18.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05d101d640 10 client.19040.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:18.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05d101d640 10 monclient: _renew_subs 2024-03-13T08:11:18.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05d101d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:18.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05d101d640 1 -- 172.21.15.132:0/2237837776 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05cc1d79a0 con 0x7f05cc141060 2024-03-13T08:11:18.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.422+0000 7f05d101d640 1 librados: init done 2024-03-13T08:11:18.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.423+0000 7f05d101d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:18.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.423+0000 7f05d101d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:18.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.423+0000 7f05d101d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:18.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.423+0000 7f05d101d640 1 -- 172.21.15.132:0/2237837776 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05cc04f000 con 0x7f05cc141060 2024-03-13T08:11:18.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.457+0000 7f05b37fe640 1 -- 172.21.15.132:0/2237837776 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f05c400ab40 con 0x7f05cc141060 2024-03-13T08:11:18.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.458+0000 7f05b37fe640 1 --2- 172.21.15.132:0/2237837776 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f05b806d5b0 0x7f05b806fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:18.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.458+0000 7f05cad9d640 1 --2- 172.21.15.132:0/2237837776 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f05b806d5b0 0x7f05b806fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:18.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.458+0000 7f05cad9d640 10 monclient: get_auth_request con 0x7f05b806d5b0 auth_method 0 2024-03-13T08:11:18.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.458+0000 7f05b37fe640 10 client.19040.objecter ms_handle_connect 0x7f05b806d5b0 2024-03-13T08:11:18.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.458+0000 7f05cad9d640 1 --2- 172.21.15.132:0/2237837776 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f05b806d5b0 0x7f05b806fa70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f05bc034000 tx=0x7f05bc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:18.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.707+0000 7f05b37fe640 1 -- 172.21.15.132:0/2237837776 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f05c400ae30 con 0x7f05cc141060 2024-03-13T08:11:18.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.707+0000 7f05b37fe640 10 client.19040.objecter ms_dispatch 0x7f05cc000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:18.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.707+0000 7f05b37fe640 3 client.19040.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:18.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.707+0000 7f05b37fe640 3 client.19040.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:18.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.708+0000 7f05b37fe640 20 client.19040.objecter dump_active .. 0 homeless 2024-03-13T08:11:18.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.708+0000 7f05b37fe640 1 -- 172.21.15.132:0/2237837776 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f05c4059120 con 0x7f05cc141060 2024-03-13T08:11:18.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.708+0000 7f05b37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:18.709 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.708+0000 7f05b37fe640 10 monclient: _finish_command 1 = system:0 2024-03-13T08:11:18.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.887+0000 7f05d101d640 1 -- 172.21.15.132:0/2237837776 --> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f05cc067e10 con 0x7f05b806d5b0 2024-03-13T08:11:18.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.887+0000 7f05b37fe640 1 -- 172.21.15.132:0/2237837776 <== mgr.4104 v2:172.21.15.132:6800/3847550149 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f05cc067e10 con 0x7f05b806d5b0 2024-03-13T08:11:18.888 INFO:teuthology.orchestra.run.smithi132.stderr:instructed osd(s) 7 to deep-scrub 2024-03-13T08:11:18.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 10 librados: watch_flush enter 2024-03-13T08:11:18.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 10 librados: watch_flush exit 2024-03-13T08:11:18.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 20 client.19040.objecter shutdown clearing up homeless session... 2024-03-13T08:11:18.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 10 client.19040.objecter successfully canceled tick 2024-03-13T08:11:18.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 1 -- 172.21.15.132:0/2237837776 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f05b806d5b0 msgr2=0x7f05b806fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:18.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 1 --2- 172.21.15.132:0/2237837776 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f05b806d5b0 0x7f05b806fa70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f05bc034000 tx=0x7f05bc008030 comp rx=0 tx=0).stop 2024-03-13T08:11:18.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 10 monclient: shutdown 2024-03-13T08:11:18.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:18.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 1 -- 172.21.15.132:0/2237837776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05cc141060 msgr2=0x7f05cc1db4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:18.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 1 --2- 172.21.15.132:0/2237837776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05cc141060 0x7f05cc1db4f0 secure :-1 s=READY pgs=3545 cs=0 l=1 rev1=1 crypto rx=0x7f05cc13fa10 tx=0x7f05c400cc00 comp rx=0 tx=0).stop 2024-03-13T08:11:18.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 1 -- 172.21.15.132:0/2237837776 shutdown_connections 2024-03-13T08:11:18.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 1 --2- 172.21.15.132:0/2237837776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05cc141060 0x7f05cc1db4f0 unknown :-1 s=CLOSED pgs=3545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 1 --2- 172.21.15.132:0/2237837776 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f05b806d5b0 0x7f05b806fa70 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 1 --2- 172.21.15.132:0/2237837776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05cc140be0 0x7f05cc1dafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 1 --2- 172.21.15.132:0/2237837776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05b40039b0 0x7f05cc0ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:18.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 1 -- 172.21.15.132:0/2237837776 >> 172.21.15.132:0/2237837776 conn(0x7f05cc06d2b0 msgr2=0x7f05cc0b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:18.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 1 -- 172.21.15.132:0/2237837776 shutdown_connections 2024-03-13T08:11:18.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 1 -- 172.21.15.132:0/2237837776 wait complete. 2024-03-13T08:11:18.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:18.889+0000 7f05d101d640 1 librados: shutdown 2024-03-13T08:11:18.904 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-13T08:11:19.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.005+0000 7fcaab2d6640 10 monclient: get_monmap_and_config 2024-03-13T08:11:19.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.005+0000 7fcaab2d6640 10 monclient: build_initial_monmap 2024-03-13T08:11:19.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.005+0000 7fcaab2d6640 10 monclient: monmap: 2024-03-13T08:11:19.005 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:19.005 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:19.005 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:19.005844+0000 2024-03-13T08:11:19.005 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:19.005844+0000 2024-03-13T08:11:19.006 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:19.006 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:19.006 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:19.006 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:19.006 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:19.006 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:19.006 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:19.006 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:19.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.005+0000 7fcaab2d6640 1 Processor -- start 2024-03-13T08:11:19.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.005+0000 7fcaab2d6640 1 -- start start 2024-03-13T08:11:19.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.005+0000 7fcaab2d6640 10 monclient: init 2024-03-13T08:11:19.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.006+0000 7fcaab2d6640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:19.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.006+0000 7fcaab2d6640 10 monclient: _add_conns ranks=[4,0,1,3,2] 2024-03-13T08:11:19.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.006+0000 7fcaab2d6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcaa40adb50 0x7fcaa413f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:19.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.006+0000 7fcaab2d6640 10 monclient(hunting): picked mon.noname-c con 0x7fcaa40adb50 addr v2:172.21.15.195:3300/0 2024-03-13T08:11:19.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.006+0000 7fcaab2d6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcaa406ecc0 addr v1:172.21.15.132:6789/0 2024-03-13T08:11:19.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.006+0000 7fcaab2d6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fcaa4142cb0 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:19.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.006+0000 7fcaab2d6640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa40b07a0 con 0x7fcaa406ecc0 2024-03-13T08:11:19.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.006+0000 7fcaab2d6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa4070000 con 0x7fcaa4142cb0 2024-03-13T08:11:19.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.006+0000 7fcaab2d6640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:19.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.006+0000 7fcaab2d6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa40adf30 con 0x7fcaa40adb50 2024-03-13T08:11:19.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.006+0000 7fcaab2d6640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:19.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.006+0000 7fcaab2d6640 10 monclient(hunting): authenticate will time out at 3506.414566s 2024-03-13T08:11:19.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.006+0000 7fcaa8871640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fcaa406ecc0 0x7fcaa41425a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:50474/0 (socket says 172.21.15.132:50474) 2024-03-13T08:11:19.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.006+0000 7fcaa8871640 1 -- 172.21.15.132:0/3778564378 learned_addr learned my addr 172.21.15.132:0/3778564378 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T08:11:19.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.007+0000 7fcaa9072640 1 --2- 172.21.15.132:0/3778564378 >> v2:172.21.15.195:3300/0 conn(0x7fcaa40adb50 0x7fcaa413f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:19.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.007+0000 7fcaa9072640 10 monclient(hunting): get_auth_request con 0x7fcaa40adb50 auth_method 0 2024-03-13T08:11:19.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.007+0000 7fcaa9072640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:19.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.007+0000 7fcaa9072640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:19.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.007+0000 7fcaa9072640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:19.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.007+0000 7fcaa9072640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:19.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.007+0000 7fcaa9072640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:19.011 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.007+0000 7fcaa9072640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:19.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.009+0000 7fcaa3fff640 1 -- 172.21.15.132:0/3778564378 <== mon.0 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374653533 0 0) 0x7fcaa40b07a0 con 0x7fcaa406ecc0 2024-03-13T08:11:19.012 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.009+0000 7fcaa3fff640 10 monclient(hunting): _negotiate 2024-03-13T08:11:19.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.009+0000 7fcaa3fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:19.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.009+0000 7fcaa3fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:19.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.009+0000 7fcaa3fff640 10 monclient(hunting): my global_id is 16098 2024-03-13T08:11:19.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.009+0000 7fcaa3fff640 1 -- 172.21.15.132:0/3778564378 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca8c003520 con 0x7fcaa406ecc0 2024-03-13T08:11:19.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.009+0000 7fcaa9072640 10 monclient(hunting): handle_auth_done global_id 19741 payload 274 2024-03-13T08:11:19.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.009+0000 7fcaa9072640 1 -- 172.21.15.132:0/3778564378 >> v1:172.21.15.195:6789/0 conn(0x7fcaa4142cb0 legacy=0x7fcaa414ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T08:11:19.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.009+0000 7fcaa9072640 1 -- 172.21.15.132:0/3778564378 >> v1:172.21.15.132:6789/0 conn(0x7fcaa406ecc0 legacy=0x7fcaa41425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:19.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.009+0000 7fcaa9072640 10 monclient: _finish_hunting 0 2024-03-13T08:11:19.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.009+0000 7fcaa9072640 1 monclient: found mon.noname-c 2024-03-13T08:11:19.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.009+0000 7fcaa9072640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:19.013 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.009+0000 7fcaa9072640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.195:3300/0 2024-03-13T08:11:19.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.009+0000 7fcaa9072640 1 -- 172.21.15.132:0/3778564378 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa406e870 con 0x7fcaa40adb50 2024-03-13T08:11:19.014 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.011+0000 7fcaa9072640 1 --2- 172.21.15.132:0/3778564378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaa40adb50 0x7fcaa413f7e0 secure :-1 s=READY pgs=4745 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0097a0 tx=0x7fca9c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a085a1264819b8d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:19.015 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaa3fff640 1 -- 172.21.15.132:0/3778564378 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c035020 con 0x7fcaa40adb50 2024-03-13T08:11:19.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaa3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:19.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaa3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T08:11:19.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaa3fff640 10 monclient: dump: 2024-03-13T08:11:19.016 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:19.016 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:19.016 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:19.016 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:19.016 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:19.016 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:19.016 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:19.016 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:19.016 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:19.016 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:19.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaa3fff640 10 monclient: _finish_auth 0 2024-03-13T08:11:19.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaa3fff640 10 monclient: _check_auth_tickets 2024-03-13T08:11:19.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaa3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:19.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaa3fff640 1 -- 172.21.15.132:0/3778564378 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c031830 con 0x7fcaa40adb50 2024-03-13T08:11:19.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaab2d6640 5 monclient: authenticate success, global_id 19741 2024-03-13T08:11:19.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaab2d6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T08:11:19.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaa3fff640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:19.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaab2d6640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:19.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaa3fff640 1 -- 172.21.15.132:0/3778564378 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c031c00 con 0x7fcaa40adb50 2024-03-13T08:11:19.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaab2d6640 4 set_mon_vals no callback set 2024-03-13T08:11:19.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaa3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:19.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaa3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-03-13T08:11:19.017 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.015+0000 7fcaa3fff640 10 monclient: dump: 2024-03-13T08:11:19.018 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:19.018 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:19.018 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:19.018 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:19.018 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:19.018 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:19.018 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:19.018 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:19.018 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:19.018 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:19.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.016+0000 7fcaab2d6640 10 monclient: shutdown 2024-03-13T08:11:19.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.016+0000 7fcaab2d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:19.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.016+0000 7fcaab2d6640 1 -- 172.21.15.132:0/3778564378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaa40adb50 msgr2=0x7fcaa413f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:19.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.016+0000 7fcaab2d6640 1 --2- 172.21.15.132:0/3778564378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaa40adb50 0x7fcaa413f7e0 secure :-1 s=READY pgs=4745 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0097a0 tx=0x7fca9c02b8d0 comp rx=0 tx=0).stop 2024-03-13T08:11:19.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.016+0000 7fcaab2d6640 1 -- 172.21.15.132:0/3778564378 shutdown_connections 2024-03-13T08:11:19.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.016+0000 7fcaab2d6640 1 --2- 172.21.15.132:0/3778564378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaa40adb50 0x7fcaa413f7e0 unknown :-1 s=CLOSED pgs=4745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:19.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.016+0000 7fcaab2d6640 1 -- 172.21.15.132:0/3778564378 >> 172.21.15.132:0/3778564378 conn(0x7fcaa40b0c80 msgr2=0x7fcaa40b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:19.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.016+0000 7fcaab2d6640 1 -- 172.21.15.132:0/3778564378 shutdown_connections 2024-03-13T08:11:19.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.016+0000 7fcaab2d6640 1 -- 172.21.15.132:0/3778564378 wait complete. 2024-03-13T08:11:19.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.017+0000 7fcaab2d6640 10 monclient: build_initial_monmap 2024-03-13T08:11:19.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.017+0000 7fcaab2d6640 10 monclient: monmap: 2024-03-13T08:11:19.023 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:19.023 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:19.023 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:19.023 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:19.023 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:19.023 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:19.023 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:19.024 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:19.024 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:19.024 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:19.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.017+0000 7fcaab2d6640 1 librados: starting msgr at 2024-03-13T08:11:19.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.017+0000 7fcaab2d6640 1 librados: starting objecter 2024-03-13T08:11:19.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.017+0000 7fcaab2d6640 1 Processor -- start 2024-03-13T08:11:19.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.017+0000 7fcaab2d6640 1 -- start start 2024-03-13T08:11:19.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.017+0000 7fcaab2d6640 1 librados: setting wanted keys 2024-03-13T08:11:19.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.017+0000 7fcaab2d6640 1 librados: calling monclient init 2024-03-13T08:11:19.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.017+0000 7fcaab2d6640 10 monclient: init 2024-03-13T08:11:19.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaab2d6640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:19.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaab2d6640 10 monclient: _add_conns ranks=[2,0,1] 2024-03-13T08:11:19.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaab2d6640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcaa40adb50 0x7fcaa40b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:19.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaab2d6640 10 monclient(hunting): picked mon.noname-c con 0x7fcaa40adb50 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:19.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaab2d6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaa4142cb0 0x7fcaa41d7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:19.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaab2d6640 10 monclient(hunting): picked mon.noname-a con 0x7fcaa4142cb0 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:19.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaab2d6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaa41438a0 0x7fcaa41d84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:19.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaab2d6640 10 monclient(hunting): picked mon.noname-b con 0x7fcaa41438a0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:19.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaab2d6640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:19.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaab2d6640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa413fd20 con 0x7fcaa4142cb0 2024-03-13T08:11:19.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaab2d6640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:19.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaab2d6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40aff10 con 0x7fcaa41438a0 2024-03-13T08:11:19.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaab2d6640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:19.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaab2d6640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa413fea0 con 0x7fcaa40adb50 2024-03-13T08:11:19.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaab2d6640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:19.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaab2d6640 10 monclient(hunting): authenticate will time out at 3506.426235s 2024-03-13T08:11:19.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaa8871640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaa4142cb0 0x7fcaa41d7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:19.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaa9873640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaa41438a0 0x7fcaa41d84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:19.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaa9072640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcaa40adb50 0x7fcaa40b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:19.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaa8871640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaa4142cb0 0x7fcaa41d7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:56250/0 (socket says 172.21.15.132:56250) 2024-03-13T08:11:19.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaa8871640 1 -- 172.21.15.132:0/3841748654 learned_addr learned my addr 172.21.15.132:0/3841748654 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:19.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaa8871640 10 monclient(hunting): get_auth_request con 0x7fcaa4142cb0 auth_method 0 2024-03-13T08:11:19.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaa8871640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:19.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaa8871640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:19.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.018+0000 7fcaa8871640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:19.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa9873640 10 monclient(hunting): get_auth_request con 0x7fcaa41438a0 auth_method 0 2024-03-13T08:11:19.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa9873640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:19.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa9873640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:19.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa9873640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:19.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa9072640 10 monclient(hunting): get_auth_request con 0x7fcaa40adb50 auth_method 0 2024-03-13T08:11:19.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa9072640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:19.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa9072640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:19.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa9072640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:19.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa8871640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:19.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa8871640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:19.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa8871640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:19.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa9873640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:19.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa9873640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:19.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa9873640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:19.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa9072640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:19.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa9072640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:19.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.019+0000 7fcaa9072640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:19.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.022+0000 7fcaa8871640 10 monclient(hunting): handle_auth_done global_id 16101 payload 824 2024-03-13T08:11:19.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.022+0000 7fcaa8871640 1 -- 172.21.15.132:0/3841748654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcaa40adb50 msgr2=0x7fcaa40b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:19.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.022+0000 7fcaa8871640 1 --2- 172.21.15.132:0/3841748654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcaa40adb50 0x7fcaa40b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:19.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.022+0000 7fcaa8871640 1 -- 172.21.15.132:0/3841748654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaa41438a0 msgr2=0x7fcaa41d84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:19.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.022+0000 7fcaa8871640 1 --2- 172.21.15.132:0/3841748654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaa41438a0 0x7fcaa41d84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:19.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.022+0000 7fcaa8871640 10 monclient: _finish_hunting 0 2024-03-13T08:11:19.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.022+0000 7fcaa8871640 1 monclient: found mon.noname-a 2024-03-13T08:11:19.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.022+0000 7fcaa8871640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:19.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.022+0000 7fcaa8871640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T08:11:19.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.022+0000 7fcaa8871640 1 -- 172.21.15.132:0/3841748654 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa4144b00 con 0x7fcaa4142cb0 2024-03-13T08:11:19.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.024+0000 7fcaa8871640 1 --2- 172.21.15.132:0/3841748654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaa4142cb0 0x7fcaa41d7fb0 secure :-1 s=READY pgs=3277 cs=0 l=1 rev1=1 crypto rx=0x7fca9800cbc0 tx=0x7fca9800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:19.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.024+0000 7fcaa1ffb640 10 client.?.objecter ms_handle_connect 0x7fcaa4142cb0 2024-03-13T08:11:19.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.024+0000 7fcaa1ffb640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:19.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaa1ffb640 1 -- 172.21.15.132:0/3841748654 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98017070 con 0x7fcaa4142cb0 2024-03-13T08:11:19.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaa1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:19.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaa1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T08:11:19.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaa1ffb640 10 monclient: dump: 2024-03-13T08:11:19.030 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:19.030 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:19.030 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:19.030 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:19.030 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:19.030 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:19.030 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:19.030 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:19.030 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:19.031 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:19.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaa1ffb640 10 monclient: _finish_auth 0 2024-03-13T08:11:19.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaa1ffb640 10 monclient: _check_auth_tickets 2024-03-13T08:11:19.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaa1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:19.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaa1ffb640 1 -- 172.21.15.132:0/3841748654 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9800ad40 con 0x7fcaa4142cb0 2024-03-13T08:11:19.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaa1ffb640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:19.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaa1ffb640 1 -- 172.21.15.132:0/3841748654 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98012600 con 0x7fcaa4142cb0 2024-03-13T08:11:19.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaab2d6640 5 monclient: authenticate success, global_id 16101 2024-03-13T08:11:19.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaa1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:19.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaa1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T08:11:19.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaa1ffb640 10 monclient: dump: 2024-03-13T08:11:19.031 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:19.031 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:19.031 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:19.031 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:19.032 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:19.032 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:19.032 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:19.032 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:19.032 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:19.032 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:19.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaab2d6640 10 monclient: _renew_subs 2024-03-13T08:11:19.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaab2d6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T08:11:19.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaab2d6640 1 -- 172.21.15.132:0/3841748654 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa40b2b70 con 0x7fcaa4142cb0 2024-03-13T08:11:19.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaa37fe640 4 set_mon_vals no callback set 2024-03-13T08:11:19.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaab2d6640 10 client.16101.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:19.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaab2d6640 10 monclient: _renew_subs 2024-03-13T08:11:19.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaab2d6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T08:11:19.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaab2d6640 1 -- 172.21.15.132:0/3841748654 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa4140c10 con 0x7fcaa4142cb0 2024-03-13T08:11:19.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.028+0000 7fcaab2d6640 1 librados: init done 2024-03-13T08:11:19.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.029+0000 7fcaab2d6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:19.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.029+0000 7fcaab2d6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:19.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.029+0000 7fcaab2d6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T08:11:19.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.029+0000 7fcaab2d6640 1 -- 172.21.15.132:0/3841748654 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa404f000 con 0x7fcaa4142cb0 2024-03-13T08:11:19.297 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.297+0000 7fcaa1ffb640 1 -- 172.21.15.132:0/3841748654 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7fca9801a070 con 0x7fcaa4142cb0 2024-03-13T08:11:19.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.297+0000 7fcaa1ffb640 1 --2- 172.21.15.132:0/3841748654 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fca9006d5b0 0x7fca9006fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:19.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.297+0000 7fcaa9072640 1 --2- 172.21.15.132:0/3841748654 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fca9006d5b0 0x7fca9006fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:19.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.297+0000 7fcaa9072640 10 monclient: get_auth_request con 0x7fca9006d5b0 auth_method 0 2024-03-13T08:11:19.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.297+0000 7fcaa9072640 1 --2- 172.21.15.132:0/3841748654 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fca9006d5b0 0x7fca9006fa70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fca9c036000 tx=0x7fca9c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:19.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.297+0000 7fcaa1ffb640 10 client.16101.objecter ms_handle_connect 0x7fca9006d5b0 2024-03-13T08:11:19.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.311+0000 7fcaa1ffb640 1 -- 172.21.15.132:0/3841748654 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7fca980572b0 con 0x7fcaa4142cb0 2024-03-13T08:11:19.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.311+0000 7fcaa1ffb640 10 client.16101.objecter ms_dispatch 0x7fcaa4000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:19.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.311+0000 7fcaa1ffb640 3 client.16101.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:19.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.311+0000 7fcaa1ffb640 3 client.16101.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:19.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.311+0000 7fcaa1ffb640 20 client.16101.objecter dump_active .. 0 homeless 2024-03-13T08:11:19.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.311+0000 7fcaa1ffb640 1 -- 172.21.15.132:0/3841748654 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7fca98059210 con 0x7fcaa4142cb0 2024-03-13T08:11:19.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.311+0000 7fcaa1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:19.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.311+0000 7fcaa1ffb640 10 monclient: _finish_command 1 = system:0 2024-03-13T08:11:19.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.498+0000 7fcaab2d6640 1 -- 172.21.15.132:0/3841748654 --> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcaa41448b0 con 0x7fca9006d5b0 2024-03-13T08:11:19.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.501+0000 7fcaa1ffb640 1 -- 172.21.15.132:0/3841748654 <== mgr.4104 v2:172.21.15.132:6800/3847550149 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+115646 (secure 0 0 0) 0x7fca9c0653d0 con 0x7fca9006d5b0 2024-03-13T08:11:19.503 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T08:11:19.504 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-03-13T08:11:19.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 10 librados: watch_flush enter 2024-03-13T08:11:19.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 10 librados: watch_flush exit 2024-03-13T08:11:19.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 20 client.16101.objecter shutdown clearing up homeless session... 2024-03-13T08:11:19.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 10 client.16101.objecter successfully canceled tick 2024-03-13T08:11:19.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 1 -- 172.21.15.132:0/3841748654 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fca9006d5b0 msgr2=0x7fca9006fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:19.506 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 1 --2- 172.21.15.132:0/3841748654 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fca9006d5b0 0x7fca9006fa70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fca9c036000 tx=0x7fca9c03c040 comp rx=0 tx=0).stop 2024-03-13T08:11:19.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 10 monclient: shutdown 2024-03-13T08:11:19.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:19.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 1 -- 172.21.15.132:0/3841748654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaa4142cb0 msgr2=0x7fcaa41d7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:19.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 1 --2- 172.21.15.132:0/3841748654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaa4142cb0 0x7fcaa41d7fb0 secure :-1 s=READY pgs=3277 cs=0 l=1 rev1=1 crypto rx=0x7fca9800cbc0 tx=0x7fca9800a430 comp rx=0 tx=0).stop 2024-03-13T08:11:19.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 1 -- 172.21.15.132:0/3841748654 shutdown_connections 2024-03-13T08:11:19.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 1 --2- 172.21.15.132:0/3841748654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaa41438a0 0x7fcaa41d84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:19.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 1 --2- 172.21.15.132:0/3841748654 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7fca9006d5b0 0x7fca9006fa70 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:19.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 1 --2- 172.21.15.132:0/3841748654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaa4142cb0 0x7fcaa41d7fb0 unknown :-1 s=CLOSED pgs=3277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:19.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 1 --2- 172.21.15.132:0/3841748654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcaa40adb50 0x7fcaa40b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:19.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 1 -- 172.21.15.132:0/3841748654 >> 172.21.15.132:0/3841748654 conn(0x7fcaa40b0c80 msgr2=0x7fcaa40b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:19.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 1 -- 172.21.15.132:0/3841748654 shutdown_connections 2024-03-13T08:11:19.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 1 -- 172.21.15.132:0/3841748654 wait complete. 2024-03-13T08:11:19.507 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:19.504+0000 7fcaab2d6640 1 librados: shutdown 2024-03-13T08:11:19.516 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":2399,"stamp":"2024-03-13T08:11:17.792122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":581752,"kb_used_data":39384,"kb_used_omap":12,"kb_used_meta":542259,"kb_avail":754392968,"statfs":{"total":773094113280,"available":772498399232,"internally_reserved":0,"allocated":40329216,"data_stored":18763674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":555273778},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":82,"apply_latency_ms":82,"commit_latency_ns":82000000,"apply_latency_ns":82000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.613104"},"pg_stats":[{"pgid":"9.19","version":"0'0","reported_seq":2498,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.329651+0000","last_change":"2024-03-13T07:35:58.099771+0000","last_active":"2024-03-13T08:11:10.329651+0000","last_peered":"2024-03-13T08:11:10.329651+0000","last_clean":"2024-03-13T08:11:10.329651+0000","last_became_active":"2024-03-13T07:35:58.062655+0000","last_became_peered":"2024-03-13T07:35:58.062655+0000","last_unstale":"2024-03-13T08:11:10.329651+0000","last_undegraded":"2024-03-13T08:11:10.329651+0000","last_fullsized":"2024-03-13T08:11:10.329651+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:18:10.709550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":2498,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:16.029752+0000","last_change":"2024-03-13T08:11:16.030524+0000","last_active":"2024-03-13T08:11:16.029752+0000","last_peered":"2024-03-13T08:11:16.029752+0000","last_clean":"2024-03-13T08:11:16.029752+0000","last_became_active":"2024-03-13T07:35:56.409964+0000","last_became_peered":"2024-03-13T07:35:56.409964+0000","last_unstale":"2024-03-13T08:11:16.029752+0000","last_undegraded":"2024-03-13T08:11:16.029752+0000","last_fullsized":"2024-03-13T08:11:16.029752+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:16.027915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:16.027915+0000","last_clean_scrub_stamp":"2024-03-13T08:11:16.027915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T16:59:29.059646+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":2489,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.331997+0000","last_change":"2024-03-13T07:35:59.115829+0000","last_active":"2024-03-13T08:11:10.331997+0000","last_peered":"2024-03-13T08:11:10.331997+0000","last_clean":"2024-03-13T08:11:10.331997+0000","last_became_active":"2024-03-13T07:35:59.109130+0000","last_became_peered":"2024-03-13T07:35:59.109130+0000","last_unstale":"2024-03-13T08:11:10.331997+0000","last_undegraded":"2024-03-13T08:11:10.331997+0000","last_fullsized":"2024-03-13T08:11:10.331997+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":2489,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.357437+0000","last_change":"2024-03-13T07:36:26.745752+0000","last_active":"2024-03-13T08:11:10.357437+0000","last_peered":"2024-03-13T08:11:10.357437+0000","last_clean":"2024-03-13T08:11:10.357437+0000","last_became_active":"2024-03-13T07:36:26.737387+0000","last_became_peered":"2024-03-13T07:36:26.737387+0000","last_unstale":"2024-03-13T08:11:10.357437+0000","last_undegraded":"2024-03-13T08:11:10.357437+0000","last_fullsized":"2024-03-13T08:11:10.357437+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:06:10.706337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":2492,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.367867+0000","last_change":"2024-03-13T07:35:56.978987+0000","last_active":"2024-03-13T08:11:10.367867+0000","last_peered":"2024-03-13T08:11:10.367867+0000","last_clean":"2024-03-13T08:11:10.367867+0000","last_became_active":"2024-03-13T07:35:56.973580+0000","last_became_peered":"2024-03-13T07:35:56.973580+0000","last_unstale":"2024-03-13T08:11:10.367867+0000","last_undegraded":"2024-03-13T08:11:10.367867+0000","last_fullsized":"2024-03-13T08:11:10.367867+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:11:56.932620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":2485,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.351170+0000","last_change":"2024-03-13T07:35:56.488106+0000","last_active":"2024-03-13T08:11:10.351170+0000","last_peered":"2024-03-13T08:11:10.351170+0000","last_clean":"2024-03-13T08:11:10.351170+0000","last_became_active":"2024-03-13T07:35:56.482614+0000","last_became_peered":"2024-03-13T07:35:56.482614+0000","last_unstale":"2024-03-13T08:11:10.351170+0000","last_undegraded":"2024-03-13T08:11:10.351170+0000","last_fullsized":"2024-03-13T08:11:10.351170+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:43:18.546715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":2500,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:17.131754+0000","last_change":"2024-03-13T08:11:17.132530+0000","last_active":"2024-03-13T08:11:17.131754+0000","last_peered":"2024-03-13T08:11:17.131754+0000","last_clean":"2024-03-13T08:11:17.131754+0000","last_became_active":"2024-03-13T07:35:56.864351+0000","last_became_peered":"2024-03-13T07:35:56.864351+0000","last_unstale":"2024-03-13T08:11:17.131754+0000","last_undegraded":"2024-03-13T08:11:17.131754+0000","last_fullsized":"2024-03-13T08:11:17.131754+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:17.130198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:17.130198+0000","last_clean_scrub_stamp":"2024-03-13T08:11:17.130198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T09:50:32.171205+0000","scrub_duration":76,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":2500,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:15.017891+0000","last_change":"2024-03-13T08:11:15.019271+0000","last_active":"2024-03-13T08:11:15.017891+0000","last_peered":"2024-03-13T08:11:15.017891+0000","last_clean":"2024-03-13T08:11:15.017891+0000","last_became_active":"2024-03-13T07:35:55.874083+0000","last_became_peered":"2024-03-13T07:35:55.874083+0000","last_unstale":"2024-03-13T08:11:15.017891+0000","last_undegraded":"2024-03-13T08:11:15.017891+0000","last_fullsized":"2024-03-13T08:11:15.017891+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:15.011783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:15.011783+0000","last_clean_scrub_stamp":"2024-03-13T08:11:15.011783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:14:28.514325+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"44'7","reported_seq":2499,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.342014+0000","last_change":"2024-03-13T07:35:58.311435+0000","last_active":"2024-03-13T08:11:10.342014+0000","last_peered":"2024-03-13T08:11:10.342014+0000","last_clean":"2024-03-13T08:11:10.342014+0000","last_became_active":"2024-03-13T07:35:58.303418+0000","last_became_peered":"2024-03-13T07:35:58.303418+0000","last_unstale":"2024-03-13T08:11:10.342014+0000","last_undegraded":"2024-03-13T08:11:10.342014+0000","last_fullsized":"2024-03-13T08:11:10.342014+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:15:47.397228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":2491,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.339418+0000","last_change":"2024-03-13T07:36:00.388143+0000","last_active":"2024-03-13T08:11:10.339418+0000","last_peered":"2024-03-13T08:11:10.339418+0000","last_clean":"2024-03-13T08:11:10.339418+0000","last_became_active":"2024-03-13T07:36:00.379147+0000","last_became_peered":"2024-03-13T07:36:00.379147+0000","last_unstale":"2024-03-13T08:11:10.339418+0000","last_undegraded":"2024-03-13T08:11:10.339418+0000","last_fullsized":"2024-03-13T08:11:10.339418+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:55:32.801526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":2528,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:13.662985+0000","last_change":"2024-03-13T08:11:13.663776+0000","last_active":"2024-03-13T08:11:13.662985+0000","last_peered":"2024-03-13T08:11:13.662985+0000","last_clean":"2024-03-13T08:11:13.662985+0000","last_became_active":"2024-03-13T07:35:55.898189+0000","last_became_peered":"2024-03-13T07:35:55.898189+0000","last_unstale":"2024-03-13T08:11:13.662985+0000","last_undegraded":"2024-03-13T08:11:13.662985+0000","last_fullsized":"2024-03-13T08:11:13.662985+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:13.661293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:13.661293+0000","last_clean_scrub_stamp":"2024-03-13T08:11:13.661293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T16:17:02.142150+0000","scrub_duration":68,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":2500,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.334223+0000","last_change":"2024-03-13T07:35:59.285198+0000","last_active":"2024-03-13T08:11:10.334223+0000","last_peered":"2024-03-13T08:11:10.334223+0000","last_clean":"2024-03-13T08:11:10.334223+0000","last_became_active":"2024-03-13T07:35:59.276875+0000","last_became_peered":"2024-03-13T07:35:59.276875+0000","last_unstale":"2024-03-13T08:11:10.334223+0000","last_undegraded":"2024-03-13T08:11:10.334223+0000","last_fullsized":"2024-03-13T08:11:10.334223+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:08:00.548538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"1293'78","reported_seq":2529,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:14.774577+0000","last_change":"2024-03-13T08:11:14.775819+0000","last_active":"2024-03-13T08:11:14.774577+0000","last_peered":"2024-03-13T08:11:14.774577+0000","last_clean":"2024-03-13T08:11:14.774577+0000","last_became_active":"2024-03-13T07:34:47.400882+0000","last_became_peered":"2024-03-13T07:34:47.400882+0000","last_unstale":"2024-03-13T08:11:14.774577+0000","last_undegraded":"2024-03-13T08:11:14.774577+0000","last_fullsized":"2024-03-13T08:11:14.774577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"1293'78","last_scrub_stamp":"2024-03-13T08:11:14.770336+0000","last_deep_scrub":"1293'78","last_deep_scrub_stamp":"2024-03-13T08:11:14.770336+0000","last_clean_scrub_stamp":"2024-03-13T08:11:14.770336+0000","objects_scrubbed":2,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T10:33:41.548222+0000","scrub_duration":109,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":2514,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.080069+0000","last_change":"2024-03-13T07:34:52.336528+0000","last_active":"2024-03-13T08:11:10.080069+0000","last_peered":"2024-03-13T08:11:10.080069+0000","last_clean":"2024-03-13T08:11:10.080069+0000","last_became_active":"2024-03-13T07:34:52.331245+0000","last_became_peered":"2024-03-13T07:34:52.331245+0000","last_unstale":"2024-03-13T08:11:10.080069+0000","last_undegraded":"2024-03-13T08:11:10.080069+0000","last_fullsized":"2024-03-13T08:11:10.080069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":2490,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.344063+0000","last_change":"2024-03-13T07:35:56.278016+0000","last_active":"2024-03-13T08:11:10.344063+0000","last_peered":"2024-03-13T08:11:10.344063+0000","last_clean":"2024-03-13T08:11:10.344063+0000","last_became_active":"2024-03-13T07:35:56.266938+0000","last_became_peered":"2024-03-13T07:35:56.266938+0000","last_unstale":"2024-03-13T08:11:10.344063+0000","last_undegraded":"2024-03-13T08:11:10.344063+0000","last_fullsized":"2024-03-13T08:11:10.344063+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:07:57.842277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2524,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:16.033580+0000","last_change":"2024-03-13T08:11:16.033580+0000","last_active":"2024-03-13T08:11:16.033580+0000","last_peered":"2024-03-13T08:11:16.033580+0000","last_clean":"2024-03-13T08:11:16.033580+0000","last_became_active":"2024-03-13T07:34:52.021695+0000","last_became_peered":"2024-03-13T07:34:52.021695+0000","last_unstale":"2024-03-13T08:11:16.033580+0000","last_undegraded":"2024-03-13T08:11:16.033580+0000","last_fullsized":"2024-03-13T08:11:16.033580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:16.030764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:16.030764+0000","last_clean_scrub_stamp":"2024-03-13T08:11:16.030764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T10:09:57.248468+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":2489,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.359389+0000","last_change":"2024-03-13T07:36:13.095056+0000","last_active":"2024-03-13T08:11:10.359389+0000","last_peered":"2024-03-13T08:11:10.359389+0000","last_clean":"2024-03-13T08:11:10.359389+0000","last_became_active":"2024-03-13T07:36:13.088822+0000","last_became_peered":"2024-03-13T07:36:13.088822+0000","last_unstale":"2024-03-13T08:11:10.359389+0000","last_undegraded":"2024-03-13T08:11:10.359389+0000","last_fullsized":"2024-03-13T08:11:10.359389+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T16:26:52.233143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2494,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.343311+0000","last_change":"2024-03-13T07:34:52.044522+0000","last_active":"2024-03-13T08:11:10.343311+0000","last_peered":"2024-03-13T08:11:10.343311+0000","last_clean":"2024-03-13T08:11:10.343311+0000","last_became_active":"2024-03-13T07:34:52.014922+0000","last_became_peered":"2024-03-13T07:34:52.014922+0000","last_unstale":"2024-03-13T08:11:10.343311+0000","last_undegraded":"2024-03-13T08:11:10.343311+0000","last_fullsized":"2024-03-13T08:11:10.343311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:59:30.188904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":2498,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.334962+0000","last_change":"2024-03-13T07:36:30.783179+0000","last_active":"2024-03-13T08:11:10.334962+0000","last_peered":"2024-03-13T08:11:10.334962+0000","last_clean":"2024-03-13T08:11:10.334962+0000","last_became_active":"2024-03-13T07:36:30.750203+0000","last_became_peered":"2024-03-13T07:36:30.750203+0000","last_unstale":"2024-03-13T08:11:10.334962+0000","last_undegraded":"2024-03-13T08:11:10.334962+0000","last_fullsized":"2024-03-13T08:11:10.334962+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":2514,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.076608+0000","last_change":"2024-03-13T07:34:52.914984+0000","last_active":"2024-03-13T08:11:10.076608+0000","last_peered":"2024-03-13T08:11:10.076608+0000","last_clean":"2024-03-13T08:11:10.076608+0000","last_became_active":"2024-03-13T07:34:52.909827+0000","last_became_peered":"2024-03-13T07:34:52.909827+0000","last_unstale":"2024-03-13T08:11:10.076608+0000","last_undegraded":"2024-03-13T08:11:10.076608+0000","last_fullsized":"2024-03-13T08:11:10.076608+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":2491,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.348818+0000","last_change":"2024-03-13T07:35:55.514723+0000","last_active":"2024-03-13T08:11:10.348818+0000","last_peered":"2024-03-13T08:11:10.348818+0000","last_clean":"2024-03-13T08:11:10.348818+0000","last_became_active":"2024-03-13T07:35:55.507183+0000","last_became_peered":"2024-03-13T07:35:55.507183+0000","last_unstale":"2024-03-13T08:11:10.348818+0000","last_undegraded":"2024-03-13T08:11:10.348818+0000","last_fullsized":"2024-03-13T08:11:10.348818+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T08:32:34.561274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2500,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.365920+0000","last_change":"2024-03-13T07:34:52.930585+0000","last_active":"2024-03-13T08:11:10.365920+0000","last_peered":"2024-03-13T08:11:10.365920+0000","last_clean":"2024-03-13T08:11:10.365920+0000","last_became_active":"2024-03-13T07:34:52.922427+0000","last_became_peered":"2024-03-13T07:34:52.922427+0000","last_unstale":"2024-03-13T08:11:10.365920+0000","last_undegraded":"2024-03-13T08:11:10.365920+0000","last_fullsized":"2024-03-13T08:11:10.365920+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T11:07:08.680289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":2491,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.349498+0000","last_change":"2024-03-13T07:36:03.434566+0000","last_active":"2024-03-13T08:11:10.349498+0000","last_peered":"2024-03-13T08:11:10.349498+0000","last_clean":"2024-03-13T08:11:10.349498+0000","last_became_active":"2024-03-13T07:36:03.416513+0000","last_became_peered":"2024-03-13T07:36:03.416513+0000","last_unstale":"2024-03-13T08:11:10.349498+0000","last_undegraded":"2024-03-13T08:11:10.349498+0000","last_fullsized":"2024-03-13T08:11:10.349498+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2513,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.098897+0000","last_change":"2024-03-13T07:34:52.133418+0000","last_active":"2024-03-13T08:11:10.098897+0000","last_peered":"2024-03-13T08:11:10.098897+0000","last_clean":"2024-03-13T08:11:10.098897+0000","last_became_active":"2024-03-13T07:34:52.128201+0000","last_became_peered":"2024-03-13T07:34:52.128201+0000","last_unstale":"2024-03-13T08:11:10.098897+0000","last_undegraded":"2024-03-13T08:11:10.098897+0000","last_fullsized":"2024-03-13T08:11:10.098897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":2491,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.338900+0000","last_change":"2024-03-13T07:35:58.694998+0000","last_active":"2024-03-13T08:11:10.338900+0000","last_peered":"2024-03-13T08:11:10.338900+0000","last_clean":"2024-03-13T08:11:10.338900+0000","last_became_active":"2024-03-13T07:35:58.686006+0000","last_became_peered":"2024-03-13T07:35:58.686006+0000","last_unstale":"2024-03-13T08:11:10.338900+0000","last_undegraded":"2024-03-13T08:11:10.338900+0000","last_fullsized":"2024-03-13T08:11:10.338900+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2499,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.347512+0000","last_change":"2024-03-13T07:34:52.429709+0000","last_active":"2024-03-13T08:11:10.347512+0000","last_peered":"2024-03-13T08:11:10.347512+0000","last_clean":"2024-03-13T08:11:10.347512+0000","last_became_active":"2024-03-13T07:34:52.404260+0000","last_became_peered":"2024-03-13T07:34:52.404260+0000","last_unstale":"2024-03-13T08:11:10.347512+0000","last_undegraded":"2024-03-13T08:11:10.347512+0000","last_fullsized":"2024-03-13T08:11:10.347512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":2503,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.095869+0000","last_change":"2024-03-13T07:36:24.360453+0000","last_active":"2024-03-13T08:11:10.095869+0000","last_peered":"2024-03-13T08:11:10.095869+0000","last_clean":"2024-03-13T08:11:10.095869+0000","last_became_active":"2024-03-13T07:36:24.325717+0000","last_became_peered":"2024-03-13T07:36:24.325717+0000","last_unstale":"2024-03-13T08:11:10.095869+0000","last_undegraded":"2024-03-13T08:11:10.095869+0000","last_fullsized":"2024-03-13T08:11:10.095869+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":2511,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.104989+0000","last_change":"2024-03-13T07:34:52.389651+0000","last_active":"2024-03-13T08:11:10.104989+0000","last_peered":"2024-03-13T08:11:10.104989+0000","last_clean":"2024-03-13T08:11:10.104989+0000","last_became_active":"2024-03-13T07:34:52.384430+0000","last_became_peered":"2024-03-13T07:34:52.384430+0000","last_unstale":"2024-03-13T08:11:10.104989+0000","last_undegraded":"2024-03-13T08:11:10.104989+0000","last_fullsized":"2024-03-13T08:11:10.104989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:34:50.561666+0000","last_clean_scrub_stamp":"2024-03-13T07:34:50.561666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":2484,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.349315+0000","last_change":"2024-03-13T07:35:57.207526+0000","last_active":"2024-03-13T08:11:10.349315+0000","last_peered":"2024-03-13T08:11:10.349315+0000","last_clean":"2024-03-13T08:11:10.349315+0000","last_became_active":"2024-03-13T07:35:57.201526+0000","last_became_peered":"2024-03-13T07:35:57.201526+0000","last_unstale":"2024-03-13T08:11:10.349315+0000","last_undegraded":"2024-03-13T08:11:10.349315+0000","last_fullsized":"2024-03-13T08:11:10.349315+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T11:45:26.599035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":2500,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.355599+0000","last_change":"2024-03-13T07:36:33.047609+0000","last_active":"2024-03-13T08:11:10.355599+0000","last_peered":"2024-03-13T08:11:10.355599+0000","last_clean":"2024-03-13T08:11:10.355599+0000","last_became_active":"2024-03-13T07:36:33.040995+0000","last_became_peered":"2024-03-13T07:36:33.040995+0000","last_unstale":"2024-03-13T08:11:10.355599+0000","last_undegraded":"2024-03-13T08:11:10.355599+0000","last_fullsized":"2024-03-13T08:11:10.355599+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:36:47.927475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":2500,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.358815+0000","last_change":"2024-03-13T07:35:59.334344+0000","last_active":"2024-03-13T08:11:10.358815+0000","last_peered":"2024-03-13T08:11:10.358815+0000","last_clean":"2024-03-13T08:11:10.358815+0000","last_became_active":"2024-03-13T07:35:59.327404+0000","last_became_peered":"2024-03-13T07:35:59.327404+0000","last_unstale":"2024-03-13T08:11:10.358815+0000","last_undegraded":"2024-03-13T08:11:10.358815+0000","last_fullsized":"2024-03-13T08:11:10.358815+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T07:51:42.128085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":2489,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.364140+0000","last_change":"2024-03-13T07:35:58.477704+0000","last_active":"2024-03-13T08:11:10.364140+0000","last_peered":"2024-03-13T08:11:10.364140+0000","last_clean":"2024-03-13T08:11:10.364140+0000","last_became_active":"2024-03-13T07:35:58.453584+0000","last_became_peered":"2024-03-13T07:35:58.453584+0000","last_unstale":"2024-03-13T08:11:10.364140+0000","last_undegraded":"2024-03-13T08:11:10.364140+0000","last_fullsized":"2024-03-13T08:11:10.364140+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T11:07:51.798068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":2498,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.345051+0000","last_change":"2024-03-13T07:35:58.672119+0000","last_active":"2024-03-13T08:11:10.345051+0000","last_peered":"2024-03-13T08:11:10.345051+0000","last_clean":"2024-03-13T08:11:10.345051+0000","last_became_active":"2024-03-13T07:35:58.650418+0000","last_became_peered":"2024-03-13T07:35:58.650418+0000","last_unstale":"2024-03-13T08:11:10.345051+0000","last_undegraded":"2024-03-13T08:11:10.345051+0000","last_fullsized":"2024-03-13T08:11:10.345051+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:00:23.900254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":2497,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.332062+0000","last_change":"2024-03-13T07:35:58.484442+0000","last_active":"2024-03-13T08:11:10.332062+0000","last_peered":"2024-03-13T08:11:10.332062+0000","last_clean":"2024-03-13T08:11:10.332062+0000","last_became_active":"2024-03-13T07:35:58.196136+0000","last_became_peered":"2024-03-13T07:35:58.196136+0000","last_unstale":"2024-03-13T08:11:10.332062+0000","last_undegraded":"2024-03-13T08:11:10.332062+0000","last_fullsized":"2024-03-13T08:11:10.332062+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":2491,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.330849+0000","last_change":"2024-03-13T07:36:02.406401+0000","last_active":"2024-03-13T08:11:10.330849+0000","last_peered":"2024-03-13T08:11:10.330849+0000","last_clean":"2024-03-13T08:11:10.330849+0000","last_became_active":"2024-03-13T07:36:01.627175+0000","last_became_peered":"2024-03-13T07:36:01.627175+0000","last_unstale":"2024-03-13T08:11:10.330849+0000","last_undegraded":"2024-03-13T08:11:10.330849+0000","last_fullsized":"2024-03-13T08:11:10.330849+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":2497,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.323903+0000","last_change":"2024-03-13T07:35:58.016424+0000","last_active":"2024-03-13T08:11:10.323903+0000","last_peered":"2024-03-13T08:11:10.323903+0000","last_clean":"2024-03-13T08:11:10.323903+0000","last_became_active":"2024-03-13T07:35:57.866652+0000","last_became_peered":"2024-03-13T07:35:57.866652+0000","last_unstale":"2024-03-13T08:11:10.323903+0000","last_undegraded":"2024-03-13T08:11:10.323903+0000","last_fullsized":"2024-03-13T08:11:10.323903+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":2503,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.069226+0000","last_change":"2024-03-13T07:35:55.780370+0000","last_active":"2024-03-13T08:11:10.069226+0000","last_peered":"2024-03-13T08:11:10.069226+0000","last_clean":"2024-03-13T08:11:10.069226+0000","last_became_active":"2024-03-13T07:35:55.763372+0000","last_became_peered":"2024-03-13T07:35:55.763372+0000","last_unstale":"2024-03-13T08:11:10.069226+0000","last_undegraded":"2024-03-13T08:11:10.069226+0000","last_fullsized":"2024-03-13T08:11:10.069226+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":2526,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:14.605043+0000","last_change":"2024-03-13T08:11:14.605806+0000","last_active":"2024-03-13T08:11:14.605043+0000","last_peered":"2024-03-13T08:11:14.605043+0000","last_clean":"2024-03-13T08:11:14.605043+0000","last_became_active":"2024-03-13T07:35:58.764470+0000","last_became_peered":"2024-03-13T07:35:58.764470+0000","last_unstale":"2024-03-13T08:11:14.605043+0000","last_undegraded":"2024-03-13T08:11:14.605043+0000","last_fullsized":"2024-03-13T08:11:14.605043+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:14.603464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:14.603464+0000","last_clean_scrub_stamp":"2024-03-13T08:11:14.603464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T19:01:12.081699+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":2490,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.347580+0000","last_change":"2024-03-13T07:36:03.235996+0000","last_active":"2024-03-13T08:11:10.347580+0000","last_peered":"2024-03-13T08:11:10.347580+0000","last_clean":"2024-03-13T08:11:10.347580+0000","last_became_active":"2024-03-13T07:36:03.227060+0000","last_became_peered":"2024-03-13T07:36:03.227060+0000","last_unstale":"2024-03-13T08:11:10.347580+0000","last_undegraded":"2024-03-13T08:11:10.347580+0000","last_fullsized":"2024-03-13T08:11:10.347580+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":2490,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.362297+0000","last_change":"2024-03-13T07:35:58.484714+0000","last_active":"2024-03-13T08:11:10.362297+0000","last_peered":"2024-03-13T08:11:10.362297+0000","last_clean":"2024-03-13T08:11:10.362297+0000","last_became_active":"2024-03-13T07:35:58.461765+0000","last_became_peered":"2024-03-13T07:35:58.461765+0000","last_unstale":"2024-03-13T08:11:10.362297+0000","last_undegraded":"2024-03-13T08:11:10.362297+0000","last_fullsized":"2024-03-13T08:11:10.362297+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:31:23.513938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":2502,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:10.093006+0000","last_change":"2024-03-13T07:36:14.250831+0000","last_active":"2024-03-13T08:11:10.093006+0000","last_peered":"2024-03-13T08:11:10.093006+0000","last_clean":"2024-03-13T08:11:10.093006+0000","last_became_active":"2024-03-13T07:36:14.238576+0000","last_became_peered":"2024-03-13T07:36:14.238576+0000","last_unstale":"2024-03-13T08:11:10.093006+0000","last_undegraded":"2024-03-13T08:11:10.093006+0000","last_fullsized":"2024-03-13T08:11:10.093006+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T07:35:52.185179+0000","last_clean_scrub_stamp":"2024-03-13T07:35:52.185179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":20,"seq":85899346359,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72096,"kb_used_data":4824,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299744,"statfs":{"total":96636764160,"available":96562937856,"internally_reserved":0,"allocated":4939776,"data_stored":2245998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[]},{"osd":0,"up_from":20,"seq":85899346360,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72096,"kb_used_data":4824,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299744,"statfs":{"total":96636764160,"available":96562937856,"internally_reserved":0,"allocated":4939776,"data_stored":2245998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[]},{"osd":1,"up_from":20,"seq":85899346361,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72496,"kb_used_data":5216,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299344,"statfs":{"total":96636764160,"available":96562528256,"internally_reserved":0,"allocated":5341184,"data_stored":2643838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[]},{"osd":4,"up_from":18,"seq":77309411773,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72104,"kb_used_data":4828,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299736,"statfs":{"total":96636764160,"available":96562929664,"internally_reserved":0,"allocated":4943872,"data_stored":2246003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719477175,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72100,"kb_used_data":4824,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299740,"statfs":{"total":96636764160,"available":96562933760,"internally_reserved":0,"allocated":4939776,"data_stored":2245998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509879,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72100,"kb_used_data":4824,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299740,"statfs":{"total":96636764160,"available":96562933760,"internally_reserved":0,"allocated":4939776,"data_stored":2245998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834575286,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72108,"kb_used_data":4828,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299732,"statfs":{"total":96636764160,"available":96562925568,"internally_reserved":0,"allocated":4943872,"data_stored":2246003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":68876747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575287,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76652,"kb_used_data":5216,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295188,"statfs":{"total":96636764160,"available":96558272512,"internally_reserved":0,"allocated":5341184,"data_stored":2643838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-13T08:11:19.522 INFO:tasks.ceph:pgid 9.19 last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.522 INFO:tasks.ceph:pgid 9.1b last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.522 INFO:tasks.ceph:pgid 9.1a last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.523 INFO:tasks.ceph:pgid 9.5 last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.523 INFO:tasks.ceph:pgid 9.4 last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.523 INFO:tasks.ceph:pgid 9.1 last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.523 INFO:tasks.ceph:pgid 9.0 last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.523 INFO:tasks.ceph:pgid 9.2 last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.523 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-03-13T07:34:50.561666+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=34, tm_sec=50, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.524 INFO:tasks.ceph:pgid 9.8 last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.524 INFO:tasks.ceph:pgid 9.b last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.524 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-03-13T07:34:50.561666+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=34, tm_sec=50, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.524 INFO:tasks.ceph:pgid 9.a last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.524 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-03-13T07:34:50.561666+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=34, tm_sec=50, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.524 INFO:tasks.ceph:pgid 9.9 last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.524 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-03-13T07:34:50.561666+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=34, tm_sec=50, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.524 INFO:tasks.ceph:pgid 9.f last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.524 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-03-13T07:34:50.561666+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=34, tm_sec=50, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.525 INFO:tasks.ceph:pgid 9.e last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.525 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-03-13T07:34:50.561666+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=34, tm_sec=50, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.525 INFO:tasks.ceph:pgid 9.d last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.525 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-03-13T07:34:50.561666+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=34, tm_sec=50, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.525 INFO:tasks.ceph:pgid 9.c last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.525 INFO:tasks.ceph:pgid 9.1e last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.525 INFO:tasks.ceph:pgid 9.1f last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.525 INFO:tasks.ceph:pgid 9.1c last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.526 INFO:tasks.ceph:pgid 9.1d last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.526 INFO:tasks.ceph:pgid 9.12 last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.526 INFO:tasks.ceph:pgid 9.13 last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.526 INFO:tasks.ceph:pgid 9.10 last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.526 INFO:tasks.ceph:pgid 9.11 last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.526 INFO:tasks.ceph:pgid 9.17 last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.526 INFO:tasks.ceph:pgid 9.14 last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.526 INFO:tasks.ceph:pgid 9.15 last_scrub_stamp 2024-03-13T07:35:52.185179+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=7, tm_min=35, tm_sec=52, tm_wday=2, tm_yday=73, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=13, tm_hour=8, tm_min=11, tm_sec=11, tm_wday=2, tm_yday=73, tm_isdst=0) 2024-03-13T08:11:19.527 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-13T08:11:20.072 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:43.962 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:20.119 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:43.989 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:20.120 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:43.991 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:20.123 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:44.013 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:20.124 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:44.014 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:20.126 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:44.016 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:20.143 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:44.013 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:20.145 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:44.035 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:20.147 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:44.037 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:25.127 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:49.016 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:25.131 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:49.021 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:25.143 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:49.033 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:25.176 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:49.046 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:25.179 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:49.049 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:25.180 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:49.070 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:25.183 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:49.073 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:25.192 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:49.062 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:25.196 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:49.086 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:35.150 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:59.039 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:35.153 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:59.043 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:35.164 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:37:59.054 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:35.201 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:59.090 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:35.202 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:59.092 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:35.203 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:59.073 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:35.204 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:59.075 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:35.213 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:37:59.102 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:35.215 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:37:59.085 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:39.527 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-13T08:11:39.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.638+0000 7f6182bcc640 10 monclient: get_monmap_and_config 2024-03-13T08:11:39.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.638+0000 7f6182bcc640 10 monclient: build_initial_monmap 2024-03-13T08:11:39.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.638+0000 7f6182bcc640 10 monclient: monmap: 2024-03-13T08:11:39.639 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:39.639 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:39.639 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:39.639339+0000 2024-03-13T08:11:39.639 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:39.639339+0000 2024-03-13T08:11:39.639 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:39.639 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:39.639 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:39.639 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:39.639 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:39.639 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:39.639 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:39.640 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:39.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.639+0000 7f6182bcc640 1 Processor -- start 2024-03-13T08:11:39.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.639+0000 7f6182bcc640 1 -- start start 2024-03-13T08:11:39.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.639+0000 7f6182bcc640 10 monclient: init 2024-03-13T08:11:39.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.639+0000 7f6182bcc640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:39.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.639+0000 7f6182bcc640 10 monclient: _add_conns ranks=[2,1,0,3,4] 2024-03-13T08:11:39.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.639+0000 7f6182bcc640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f617c0b86c0 0x7f617c13f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:39.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.639+0000 7f6182bcc640 10 monclient(hunting): picked mon.noname-b con 0x7f617c0b86c0 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:39.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.639+0000 7f6182bcc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f617c0b2390 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:39.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.639+0000 7f6182bcc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f617c13fe90 addr v1:172.21.15.132:6789/0 2024-03-13T08:11:39.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.640+0000 7f6182bcc640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f617c0ab0a0 con 0x7f617c13fe90 2024-03-13T08:11:39.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.640+0000 7f6182bcc640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f617c0ab380 con 0x7f617c0b2390 2024-03-13T08:11:39.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.640+0000 7f6182bcc640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:39.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.640+0000 7f6182bcc640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617c0b8aa0 con 0x7f617c0b86c0 2024-03-13T08:11:39.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.640+0000 7f6182bcc640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:39.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.640+0000 7f6182bcc640 10 monclient(hunting): authenticate will time out at 3527.048119s 2024-03-13T08:11:39.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.640+0000 7f6181169640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f617c13fe90 0x7f617c1462f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:35816/0 (socket says 172.21.15.132:35816) 2024-03-13T08:11:39.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.640+0000 7f6181169640 1 -- 172.21.15.132:0/1914333418 learned_addr learned my addr 172.21.15.132:0/1914333418 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T08:11:39.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.640+0000 7f6180968640 1 --2- 172.21.15.132:0/1914333418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f617c0b86c0 0x7f617c13f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:39.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.641+0000 7f6180968640 10 monclient(hunting): get_auth_request con 0x7f617c0b86c0 auth_method 0 2024-03-13T08:11:39.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.641+0000 7f6180968640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:39.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.641+0000 7f6180968640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:39.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.641+0000 7f6180968640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:39.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.641+0000 7f6180968640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:39.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.641+0000 7f6180968640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:39.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.641+0000 7f6180968640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:39.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.642+0000 7f617b7fe640 1 -- 172.21.15.132:0/1914333418 <== mon.0 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3821866809 0 0) 0x7f617c0ab0a0 con 0x7f617c13fe90 2024-03-13T08:11:39.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.642+0000 7f617b7fe640 10 monclient(hunting): _negotiate 2024-03-13T08:11:39.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.642+0000 7f617b7fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:39.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.642+0000 7f617b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:39.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.643+0000 7f617b7fe640 10 monclient(hunting): my global_id is 16104 2024-03-13T08:11:39.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.643+0000 7f617b7fe640 1 -- 172.21.15.132:0/1914333418 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f616c003520 con 0x7f617c13fe90 2024-03-13T08:11:39.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.643+0000 7f6180968640 10 monclient(hunting): handle_auth_done global_id 19046 payload 274 2024-03-13T08:11:39.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.643+0000 7f6180968640 1 -- 172.21.15.132:0/1914333418 >> v1:172.21.15.195:6789/0 conn(0x7f617c0b2390 legacy=0x7f617c145be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:39.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.643+0000 7f6180968640 1 -- 172.21.15.132:0/1914333418 >> v1:172.21.15.132:6789/0 conn(0x7f617c13fe90 legacy=0x7f617c1462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:39.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.643+0000 7f6180968640 10 monclient: _finish_hunting 0 2024-03-13T08:11:39.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.643+0000 7f6180968640 1 monclient: found mon.noname-b 2024-03-13T08:11:39.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.643+0000 7f6180968640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:39.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.643+0000 7f6180968640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.132:3301/0 2024-03-13T08:11:39.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.643+0000 7f6180968640 1 -- 172.21.15.132:0/1914333418 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c0ab660 con 0x7f617c0b86c0 2024-03-13T08:11:39.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.645+0000 7f6180968640 1 --2- 172.21.15.132:0/1914333418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f617c0b86c0 0x7f617c13f950 secure :-1 s=READY pgs=3546 cs=0 l=1 rev1=1 crypto rx=0x7f61700097d0 tx=0x7f617002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b356e1614c74cfd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:39.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f617b7fe640 1 -- 172.21.15.132:0/1914333418 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6170035020 con 0x7f617c0b86c0 2024-03-13T08:11:39.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f617b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:39.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f617b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-03-13T08:11:39.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f617b7fe640 10 monclient: dump: 2024-03-13T08:11:39.650 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:39.650 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:39.650 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:39.650 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:39.650 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:39.650 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:39.650 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:39.650 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:39.650 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:39.650 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:39.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f617b7fe640 10 monclient: _finish_auth 0 2024-03-13T08:11:39.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f617b7fe640 10 monclient: _check_auth_tickets 2024-03-13T08:11:39.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f617b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:39.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f6182bcc640 5 monclient: authenticate success, global_id 19046 2024-03-13T08:11:39.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f617b7fe640 1 -- 172.21.15.132:0/1914333418 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6170031890 con 0x7f617c0b86c0 2024-03-13T08:11:39.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f6182bcc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-03-13T08:11:39.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f617b7fe640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:39.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f6182bcc640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:39.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f6182bcc640 4 set_mon_vals no callback set 2024-03-13T08:11:39.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f617b7fe640 1 -- 172.21.15.132:0/1914333418 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6170031c60 con 0x7f617c0b86c0 2024-03-13T08:11:39.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f617b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:39.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f617b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T08:11:39.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.648+0000 7f617b7fe640 10 monclient: dump: 2024-03-13T08:11:39.651 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:39.651 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:39.652 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:39.652 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:39.652 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:39.652 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:39.652 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:39.652 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:39.652 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:39.652 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:39.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.649+0000 7f6182bcc640 10 monclient: shutdown 2024-03-13T08:11:39.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.649+0000 7f6182bcc640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:39.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.649+0000 7f6182bcc640 1 -- 172.21.15.132:0/1914333418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f617c0b86c0 msgr2=0x7f617c13f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:39.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.649+0000 7f6182bcc640 1 --2- 172.21.15.132:0/1914333418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f617c0b86c0 0x7f617c13f950 secure :-1 s=READY pgs=3546 cs=0 l=1 rev1=1 crypto rx=0x7f61700097d0 tx=0x7f617002b930 comp rx=0 tx=0).stop 2024-03-13T08:11:39.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.649+0000 7f6182bcc640 1 -- 172.21.15.132:0/1914333418 shutdown_connections 2024-03-13T08:11:39.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.649+0000 7f6182bcc640 1 --2- 172.21.15.132:0/1914333418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f617c0b86c0 0x7f617c13f950 unknown :-1 s=CLOSED pgs=3546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:39.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.649+0000 7f6182bcc640 1 -- 172.21.15.132:0/1914333418 >> 172.21.15.132:0/1914333418 conn(0x7f617c066e20 msgr2=0x7f617c067230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:39.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.649+0000 7f6182bcc640 1 -- 172.21.15.132:0/1914333418 shutdown_connections 2024-03-13T08:11:39.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.649+0000 7f6182bcc640 1 -- 172.21.15.132:0/1914333418 wait complete. 2024-03-13T08:11:39.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.650+0000 7f6182bcc640 10 monclient: build_initial_monmap 2024-03-13T08:11:39.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.650+0000 7f6182bcc640 10 monclient: monmap: 2024-03-13T08:11:39.656 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:39.656 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:39.656 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:39.656 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:39.656 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:39.656 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:39.657 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:39.657 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:39.657 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:39.657 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:39.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.650+0000 7f6182bcc640 1 librados: starting msgr at 2024-03-13T08:11:39.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.650+0000 7f6182bcc640 1 librados: starting objecter 2024-03-13T08:11:39.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.650+0000 7f6182bcc640 1 Processor -- start 2024-03-13T08:11:39.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.650+0000 7f6182bcc640 1 -- start start 2024-03-13T08:11:39.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.650+0000 7f6182bcc640 1 librados: setting wanted keys 2024-03-13T08:11:39.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.650+0000 7f6182bcc640 1 librados: calling monclient init 2024-03-13T08:11:39.658 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.650+0000 7f6182bcc640 10 monclient: init 2024-03-13T08:11:39.658 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.651+0000 7f6182bcc640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:39.658 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.651+0000 7f6182bcc640 10 monclient: _add_conns ranks=[0,1,2] 2024-03-13T08:11:39.658 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.651+0000 7f6182bcc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f617c0b2390 0x7f617c134900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:39.658 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.651+0000 7f6182bcc640 10 monclient(hunting): picked mon.noname-a con 0x7f617c0b2390 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:39.658 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.651+0000 7f6182bcc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f617c0b86c0 0x7f617c145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:39.658 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.651+0000 7f6182bcc640 10 monclient(hunting): picked mon.noname-b con 0x7f617c0b86c0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:39.658 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.651+0000 7f6182bcc640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f617c13fe90 0x7f617c0b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:39.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.651+0000 7f6182bcc640 10 monclient(hunting): picked mon.noname-c con 0x7f617c13fe90 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:39.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.651+0000 7f6182bcc640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:39.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.651+0000 7f6182bcc640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617c14df60 con 0x7f617c0b2390 2024-03-13T08:11:39.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.651+0000 7f6182bcc640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:39.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.651+0000 7f6182bcc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617c140340 con 0x7f617c0b86c0 2024-03-13T08:11:39.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.651+0000 7f6182bcc640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:39.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.651+0000 7f6182bcc640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617c14e0e0 con 0x7f617c13fe90 2024-03-13T08:11:39.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.651+0000 7f6182bcc640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:39.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.651+0000 7f6182bcc640 10 monclient(hunting): authenticate will time out at 3527.059717s 2024-03-13T08:11:39.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f6180968640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f617c0b2390 0x7f617c134900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:39.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f6181169640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f617c13fe90 0x7f617c0b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:39.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f617bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f617c0b86c0 0x7f617c145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:39.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f6180968640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f617c0b2390 0x7f617c134900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36952/0 (socket says 172.21.15.132:36952) 2024-03-13T08:11:39.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f6180968640 1 -- 172.21.15.132:0/3404617526 learned_addr learned my addr 172.21.15.132:0/3404617526 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:39.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f6180968640 10 monclient(hunting): get_auth_request con 0x7f617c0b2390 auth_method 0 2024-03-13T08:11:39.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f6180968640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:39.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f6180968640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:39.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f6180968640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:39.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f6181169640 10 monclient(hunting): get_auth_request con 0x7f617c13fe90 auth_method 0 2024-03-13T08:11:39.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f6181169640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:39.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f6181169640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:39.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f6181169640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:39.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f617bfff640 10 monclient(hunting): get_auth_request con 0x7f617c0b86c0 auth_method 0 2024-03-13T08:11:39.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f617bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:39.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f617bfff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:39.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f617bfff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:39.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f6181169640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:39.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.652+0000 7f6181169640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:39.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.653+0000 7f6181169640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:39.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.653+0000 7f6180968640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:39.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.653+0000 7f6180968640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:39.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.653+0000 7f6180968640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:39.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.653+0000 7f617bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:39.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.653+0000 7f617bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:39.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.653+0000 7f617bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:39.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.655+0000 7f6181169640 10 monclient(hunting): handle_auth_done global_id 19049 payload 824 2024-03-13T08:11:39.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.655+0000 7f6181169640 1 -- 172.21.15.132:0/3404617526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f617c0b86c0 msgr2=0x7f617c145440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:39.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.655+0000 7f6181169640 1 --2- 172.21.15.132:0/3404617526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f617c0b86c0 0x7f617c145440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:39.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.655+0000 7f6181169640 1 -- 172.21.15.132:0/3404617526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f617c0b2390 msgr2=0x7f617c134900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:39.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.655+0000 7f6181169640 1 --2- 172.21.15.132:0/3404617526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f617c0b2390 0x7f617c134900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:39.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.655+0000 7f6181169640 10 monclient: _finish_hunting 0 2024-03-13T08:11:39.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.655+0000 7f6181169640 1 monclient: found mon.noname-c 2024-03-13T08:11:39.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.655+0000 7f6181169640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:39.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.655+0000 7f6181169640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.132:3301/0 2024-03-13T08:11:39.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.655+0000 7f6181169640 1 -- 172.21.15.132:0/3404617526 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c0b66c0 con 0x7f617c13fe90 2024-03-13T08:11:39.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.658+0000 7f6181169640 1 --2- 172.21.15.132:0/3404617526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f617c13fe90 0x7f617c0b70a0 secure :-1 s=READY pgs=3547 cs=0 l=1 rev1=1 crypto rx=0x7f617400ca80 tx=0x7f617400cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:39.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.658+0000 7f61797fa640 10 client.?.objecter ms_handle_connect 0x7f617c13fe90 2024-03-13T08:11:39.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.658+0000 7f61797fa640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:39.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f61797fa640 1 -- 172.21.15.132:0/3404617526 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174017070 con 0x7f617c13fe90 2024-03-13T08:11:39.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f61797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:39.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f61797fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-13T08:11:39.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f61797fa640 10 monclient: dump: 2024-03-13T08:11:39.666 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:39.666 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:39.666 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:39.666 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:39.667 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:39.667 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:39.667 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:39.667 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:39.667 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:39.667 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:39.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f61797fa640 10 monclient: _finish_auth 0 2024-03-13T08:11:39.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f61797fa640 10 monclient: _check_auth_tickets 2024-03-13T08:11:39.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f61797fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:39.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f61797fa640 1 -- 172.21.15.132:0/3404617526 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617400ac30 con 0x7f617c13fe90 2024-03-13T08:11:39.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f6182bcc640 5 monclient: authenticate success, global_id 19049 2024-03-13T08:11:39.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f6182bcc640 10 monclient: _renew_subs 2024-03-13T08:11:39.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f6182bcc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:39.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f6182bcc640 1 -- 172.21.15.132:0/3404617526 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f617c0b69a0 con 0x7f617c13fe90 2024-03-13T08:11:39.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f61797fa640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:39.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f61797fa640 1 -- 172.21.15.132:0/3404617526 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6174012600 con 0x7f617c13fe90 2024-03-13T08:11:39.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f61797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:39.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f61797fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-13T08:11:39.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f61797fa640 10 monclient: dump: 2024-03-13T08:11:39.669 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:39.669 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:39.669 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:39.669 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:39.669 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:39.669 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:39.669 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:39.669 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:39.669 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:39.669 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:39.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.661+0000 7f617affd640 4 set_mon_vals no callback set 2024-03-13T08:11:39.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.662+0000 7f6182bcc640 10 client.19049.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:39.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.662+0000 7f6182bcc640 10 monclient: _renew_subs 2024-03-13T08:11:39.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.662+0000 7f6182bcc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:39.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.662+0000 7f6182bcc640 1 -- 172.21.15.132:0/3404617526 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617c145980 con 0x7f617c13fe90 2024-03-13T08:11:39.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.662+0000 7f6182bcc640 1 librados: init done 2024-03-13T08:11:39.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.662+0000 7f6182bcc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:39.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.662+0000 7f6182bcc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:39.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.662+0000 7f6182bcc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.132:3301/0 2024-03-13T08:11:39.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.662+0000 7f6182bcc640 1 -- 172.21.15.132:0/3404617526 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617c144a60 con 0x7f617c13fe90 2024-03-13T08:11:39.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.680+0000 7f61797fa640 1 -- 172.21.15.132:0/3404617526 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f617401a070 con 0x7f617c13fe90 2024-03-13T08:11:39.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.680+0000 7f61797fa640 1 --2- 172.21.15.132:0/3404617526 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f616006d5e0 0x7f616006faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:39.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.681+0000 7f6180968640 1 --2- 172.21.15.132:0/3404617526 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f616006d5e0 0x7f616006faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:39.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.681+0000 7f6180968640 10 monclient: get_auth_request con 0x7f616006d5e0 auth_method 0 2024-03-13T08:11:39.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.681+0000 7f6180968640 1 --2- 172.21.15.132:0/3404617526 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f616006d5e0 0x7f616006faa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f617002b970 tx=0x7f6170008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:39.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.681+0000 7f61797fa640 10 client.19049.objecter ms_handle_connect 0x7f616006d5e0 2024-03-13T08:11:39.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.683+0000 7f61797fa640 1 -- 172.21.15.132:0/3404617526 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f6174057120 con 0x7f617c13fe90 2024-03-13T08:11:39.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.683+0000 7f61797fa640 10 client.19049.objecter ms_dispatch 0x7f617c000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:39.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.683+0000 7f61797fa640 3 client.19049.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:39.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.683+0000 7f61797fa640 3 client.19049.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:39.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.683+0000 7f61797fa640 20 client.19049.objecter dump_active .. 0 homeless 2024-03-13T08:11:39.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.922+0000 7f61797fa640 1 -- 172.21.15.132:0/3404617526 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f617400ae20 con 0x7f617c13fe90 2024-03-13T08:11:39.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.922+0000 7f61797fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:39.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:39.922+0000 7f61797fa640 10 monclient: _finish_command 1 = system:0 2024-03-13T08:11:40.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.094+0000 7f6182bcc640 1 -- 172.21.15.132:0/3404617526 --> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f617c04f000 con 0x7f616006d5e0 2024-03-13T08:11:40.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.097+0000 7f61797fa640 1 -- 172.21.15.132:0/3404617526 <== mgr.4104 v2:172.21.15.132:6800/3847550149 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+116157 (secure 0 0 0) 0x7f617c04f000 con 0x7f616006d5e0 2024-03-13T08:11:40.099 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-13T08:11:40.099 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-03-13T08:11:40.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.099+0000 7f6182bcc640 10 librados: watch_flush enter 2024-03-13T08:11:40.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.099+0000 7f6182bcc640 10 librados: watch_flush exit 2024-03-13T08:11:40.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.099+0000 7f6182bcc640 20 client.19049.objecter shutdown clearing up homeless session... 2024-03-13T08:11:40.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.099+0000 7f6182bcc640 10 client.19049.objecter successfully canceled tick 2024-03-13T08:11:40.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.099+0000 7f6182bcc640 1 -- 172.21.15.132:0/3404617526 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f616006d5e0 msgr2=0x7f616006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:40.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.099+0000 7f6182bcc640 1 --2- 172.21.15.132:0/3404617526 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f616006d5e0 0x7f616006faa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f617002b970 tx=0x7f6170008030 comp rx=0 tx=0).stop 2024-03-13T08:11:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.099+0000 7f6182bcc640 10 monclient: shutdown 2024-03-13T08:11:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.099+0000 7f6182bcc640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.099+0000 7f6182bcc640 1 -- 172.21.15.132:0/3404617526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f617c13fe90 msgr2=0x7f617c0b70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.099+0000 7f6182bcc640 1 --2- 172.21.15.132:0/3404617526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f617c13fe90 0x7f617c0b70a0 secure :-1 s=READY pgs=3547 cs=0 l=1 rev1=1 crypto rx=0x7f617400ca80 tx=0x7f617400cf40 comp rx=0 tx=0).stop 2024-03-13T08:11:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.100+0000 7f6182bcc640 1 -- 172.21.15.132:0/3404617526 shutdown_connections 2024-03-13T08:11:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.100+0000 7f6182bcc640 1 --2- 172.21.15.132:0/3404617526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f617c13fe90 0x7f617c0b70a0 unknown :-1 s=CLOSED pgs=3547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.100+0000 7f6182bcc640 1 --2- 172.21.15.132:0/3404617526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f617c0b86c0 0x7f617c145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.100+0000 7f6182bcc640 1 --2- 172.21.15.132:0/3404617526 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f616006d5e0 0x7f616006faa0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.100+0000 7f6182bcc640 1 --2- 172.21.15.132:0/3404617526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f617c0b2390 0x7f617c134900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.100+0000 7f6182bcc640 1 -- 172.21.15.132:0/3404617526 >> 172.21.15.132:0/3404617526 conn(0x7f617c066e20 msgr2=0x7f617c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.100+0000 7f6182bcc640 1 -- 172.21.15.132:0/3404617526 shutdown_connections 2024-03-13T08:11:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.100+0000 7f6182bcc640 1 -- 172.21.15.132:0/3404617526 wait complete. 2024-03-13T08:11:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.100+0000 7f6182bcc640 1 librados: shutdown 2024-03-13T08:11:40.114 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":2410,"stamp":"2024-03-13T08:11:39.796694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459285,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":88,"num_write":120,"num_write_kb":1379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":581720,"kb_used_data":39384,"kb_used_omap":12,"kb_used_meta":542259,"kb_avail":754393000,"statfs":{"total":773094113280,"available":772498432000,"internally_reserved":0,"allocated":40329216,"data_stored":18763674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":555273778},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002622"},"pg_stats":[{"pgid":"9.19","version":"0'0","reported_seq":2507,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:18.426859+0000","last_change":"2024-03-13T08:11:18.427657+0000","last_active":"2024-03-13T08:11:18.426859+0000","last_peered":"2024-03-13T08:11:18.426859+0000","last_clean":"2024-03-13T08:11:18.426859+0000","last_became_active":"2024-03-13T07:35:58.062655+0000","last_became_peered":"2024-03-13T07:35:58.062655+0000","last_unstale":"2024-03-13T08:11:18.426859+0000","last_undegraded":"2024-03-13T08:11:18.426859+0000","last_fullsized":"2024-03-13T08:11:18.426859+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:18.425254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:18.425254+0000","last_clean_scrub_stamp":"2024-03-13T08:11:18.425254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T19:10:44.137501+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":2498,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:16.029752+0000","last_change":"2024-03-13T08:11:16.030524+0000","last_active":"2024-03-13T08:11:16.029752+0000","last_peered":"2024-03-13T08:11:16.029752+0000","last_clean":"2024-03-13T08:11:16.029752+0000","last_became_active":"2024-03-13T07:35:56.409964+0000","last_became_peered":"2024-03-13T07:35:56.409964+0000","last_unstale":"2024-03-13T08:11:16.029752+0000","last_undegraded":"2024-03-13T08:11:16.029752+0000","last_fullsized":"2024-03-13T08:11:16.029752+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:16.027915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:16.027915+0000","last_clean_scrub_stamp":"2024-03-13T08:11:16.027915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T16:59:29.059646+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":2497,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:18.456366+0000","last_change":"2024-03-13T08:11:18.457110+0000","last_active":"2024-03-13T08:11:18.456366+0000","last_peered":"2024-03-13T08:11:18.456366+0000","last_clean":"2024-03-13T08:11:18.456366+0000","last_became_active":"2024-03-13T07:35:59.109130+0000","last_became_peered":"2024-03-13T07:35:59.109130+0000","last_unstale":"2024-03-13T08:11:18.456366+0000","last_undegraded":"2024-03-13T08:11:18.456366+0000","last_fullsized":"2024-03-13T08:11:18.456366+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:18.454859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:18.454859+0000","last_clean_scrub_stamp":"2024-03-13T08:11:18.454859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:54:33.662491+0000","scrub_duration":90,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":2498,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:25.689822+0000","last_change":"2024-03-13T08:11:25.690618+0000","last_active":"2024-03-13T08:11:25.689822+0000","last_peered":"2024-03-13T08:11:25.689822+0000","last_clean":"2024-03-13T08:11:25.689822+0000","last_became_active":"2024-03-13T07:36:26.737387+0000","last_became_peered":"2024-03-13T07:36:26.737387+0000","last_unstale":"2024-03-13T08:11:25.689822+0000","last_undegraded":"2024-03-13T08:11:25.689822+0000","last_fullsized":"2024-03-13T08:11:25.689822+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:25.688253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:25.688253+0000","last_clean_scrub_stamp":"2024-03-13T08:11:25.688253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:11:43.500960+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":2502,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:20.696670+0000","last_change":"2024-03-13T08:11:20.696670+0000","last_active":"2024-03-13T08:11:20.696670+0000","last_peered":"2024-03-13T08:11:20.696670+0000","last_clean":"2024-03-13T08:11:20.696670+0000","last_became_active":"2024-03-13T07:35:56.973580+0000","last_became_peered":"2024-03-13T07:35:56.973580+0000","last_unstale":"2024-03-13T08:11:20.696670+0000","last_undegraded":"2024-03-13T08:11:20.696670+0000","last_fullsized":"2024-03-13T08:11:20.696670+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:20.694201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:20.694201+0000","last_clean_scrub_stamp":"2024-03-13T08:11:20.694201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T20:02:50.572929+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":2495,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:20.498267+0000","last_change":"2024-03-13T08:11:20.498267+0000","last_active":"2024-03-13T08:11:20.498267+0000","last_peered":"2024-03-13T08:11:20.498267+0000","last_clean":"2024-03-13T08:11:20.498267+0000","last_became_active":"2024-03-13T07:35:56.482614+0000","last_became_peered":"2024-03-13T07:35:56.482614+0000","last_unstale":"2024-03-13T08:11:20.498267+0000","last_undegraded":"2024-03-13T08:11:20.498267+0000","last_fullsized":"2024-03-13T08:11:20.498267+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:20.495920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:20.495920+0000","last_clean_scrub_stamp":"2024-03-13T08:11:20.495920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T19:41:01.456158+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":2500,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:17.131754+0000","last_change":"2024-03-13T08:11:17.132530+0000","last_active":"2024-03-13T08:11:17.131754+0000","last_peered":"2024-03-13T08:11:17.131754+0000","last_clean":"2024-03-13T08:11:17.131754+0000","last_became_active":"2024-03-13T07:35:56.864351+0000","last_became_peered":"2024-03-13T07:35:56.864351+0000","last_unstale":"2024-03-13T08:11:17.131754+0000","last_undegraded":"2024-03-13T08:11:17.131754+0000","last_fullsized":"2024-03-13T08:11:17.131754+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:17.130198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:17.130198+0000","last_clean_scrub_stamp":"2024-03-13T08:11:17.130198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T09:50:32.171205+0000","scrub_duration":76,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":2500,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:15.017891+0000","last_change":"2024-03-13T08:11:15.019271+0000","last_active":"2024-03-13T08:11:15.017891+0000","last_peered":"2024-03-13T08:11:15.017891+0000","last_clean":"2024-03-13T08:11:15.017891+0000","last_became_active":"2024-03-13T07:35:55.874083+0000","last_became_peered":"2024-03-13T07:35:55.874083+0000","last_unstale":"2024-03-13T08:11:15.017891+0000","last_undegraded":"2024-03-13T08:11:15.017891+0000","last_fullsized":"2024-03-13T08:11:15.017891+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:15.011783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:15.011783+0000","last_clean_scrub_stamp":"2024-03-13T08:11:15.011783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:14:28.514325+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"1293'8","reported_seq":2509,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:22.601150+0000","last_change":"2024-03-13T08:11:22.602040+0000","last_active":"2024-03-13T08:11:22.601150+0000","last_peered":"2024-03-13T08:11:22.601150+0000","last_clean":"2024-03-13T08:11:22.601150+0000","last_became_active":"2024-03-13T07:35:58.303418+0000","last_became_peered":"2024-03-13T07:35:58.303418+0000","last_unstale":"2024-03-13T08:11:22.601150+0000","last_undegraded":"2024-03-13T08:11:22.601150+0000","last_fullsized":"2024-03-13T08:11:22.601150+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"1293'8","last_scrub_stamp":"2024-03-13T08:11:22.599604+0000","last_deep_scrub":"1293'8","last_deep_scrub_stamp":"2024-03-13T08:11:22.599604+0000","last_clean_scrub_stamp":"2024-03-13T08:11:22.599604+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:06:27.489620+0000","scrub_duration":82,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":2500,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:23.685602+0000","last_change":"2024-03-13T08:11:23.686349+0000","last_active":"2024-03-13T08:11:23.685602+0000","last_peered":"2024-03-13T08:11:23.685602+0000","last_clean":"2024-03-13T08:11:23.685602+0000","last_became_active":"2024-03-13T07:36:00.379147+0000","last_became_peered":"2024-03-13T07:36:00.379147+0000","last_unstale":"2024-03-13T08:11:23.685602+0000","last_undegraded":"2024-03-13T08:11:23.685602+0000","last_fullsized":"2024-03-13T08:11:23.685602+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:23.684094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:23.684094+0000","last_clean_scrub_stamp":"2024-03-13T08:11:23.684094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T16:07:00.251593+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":2529,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:13.663776+0000","last_change":"2024-03-13T08:11:13.663776+0000","last_active":"2024-03-13T08:11:13.663776+0000","last_peered":"2024-03-13T08:11:13.663776+0000","last_clean":"2024-03-13T08:11:13.663776+0000","last_became_active":"2024-03-13T07:35:55.898189+0000","last_became_peered":"2024-03-13T07:35:55.898189+0000","last_unstale":"2024-03-13T08:11:13.663776+0000","last_undegraded":"2024-03-13T08:11:13.663776+0000","last_fullsized":"2024-03-13T08:11:13.663776+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:13.661293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:13.661293+0000","last_clean_scrub_stamp":"2024-03-13T08:11:13.661293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T16:17:02.142150+0000","scrub_duration":68,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":2510,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:20.449976+0000","last_change":"2024-03-13T08:11:20.449976+0000","last_active":"2024-03-13T08:11:20.449976+0000","last_peered":"2024-03-13T08:11:20.449976+0000","last_clean":"2024-03-13T08:11:20.449976+0000","last_became_active":"2024-03-13T07:35:59.276875+0000","last_became_peered":"2024-03-13T07:35:59.276875+0000","last_unstale":"2024-03-13T08:11:20.449976+0000","last_undegraded":"2024-03-13T08:11:20.449976+0000","last_fullsized":"2024-03-13T08:11:20.449976+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:20.447605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:20.447605+0000","last_clean_scrub_stamp":"2024-03-13T08:11:20.447605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T19:47:20.935879+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"1293'78","reported_seq":2529,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:14.774577+0000","last_change":"2024-03-13T08:11:14.775819+0000","last_active":"2024-03-13T08:11:14.774577+0000","last_peered":"2024-03-13T08:11:14.774577+0000","last_clean":"2024-03-13T08:11:14.774577+0000","last_became_active":"2024-03-13T07:34:47.400882+0000","last_became_peered":"2024-03-13T07:34:47.400882+0000","last_unstale":"2024-03-13T08:11:14.774577+0000","last_undegraded":"2024-03-13T08:11:14.774577+0000","last_fullsized":"2024-03-13T08:11:14.774577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"1293'78","last_scrub_stamp":"2024-03-13T08:11:14.770336+0000","last_deep_scrub":"1293'78","last_deep_scrub_stamp":"2024-03-13T08:11:14.770336+0000","last_clean_scrub_stamp":"2024-03-13T08:11:14.770336+0000","objects_scrubbed":2,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T10:33:41.548222+0000","scrub_duration":109,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":2522,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:18.124118+0000","last_change":"2024-03-13T08:11:18.124966+0000","last_active":"2024-03-13T08:11:18.124118+0000","last_peered":"2024-03-13T08:11:18.124118+0000","last_clean":"2024-03-13T08:11:18.124118+0000","last_became_active":"2024-03-13T07:34:52.331245+0000","last_became_peered":"2024-03-13T07:34:52.331245+0000","last_unstale":"2024-03-13T08:11:18.124118+0000","last_undegraded":"2024-03-13T08:11:18.124118+0000","last_fullsized":"2024-03-13T08:11:18.124118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:18.122544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:18.122544+0000","last_clean_scrub_stamp":"2024-03-13T08:11:18.122544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T10:18:03.418392+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":2499,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:19.648339+0000","last_change":"2024-03-13T08:11:19.649084+0000","last_active":"2024-03-13T08:11:19.648339+0000","last_peered":"2024-03-13T08:11:19.648339+0000","last_clean":"2024-03-13T08:11:19.648339+0000","last_became_active":"2024-03-13T07:35:56.266938+0000","last_became_peered":"2024-03-13T07:35:56.266938+0000","last_unstale":"2024-03-13T08:11:19.648339+0000","last_undegraded":"2024-03-13T08:11:19.648339+0000","last_fullsized":"2024-03-13T08:11:19.648339+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:19.646788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:19.646788+0000","last_clean_scrub_stamp":"2024-03-13T08:11:19.646788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:08:34.692581+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2524,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:16.033580+0000","last_change":"2024-03-13T08:11:16.033580+0000","last_active":"2024-03-13T08:11:16.033580+0000","last_peered":"2024-03-13T08:11:16.033580+0000","last_clean":"2024-03-13T08:11:16.033580+0000","last_became_active":"2024-03-13T07:34:52.021695+0000","last_became_peered":"2024-03-13T07:34:52.021695+0000","last_unstale":"2024-03-13T08:11:16.033580+0000","last_undegraded":"2024-03-13T08:11:16.033580+0000","last_fullsized":"2024-03-13T08:11:16.033580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:16.030764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:16.030764+0000","last_clean_scrub_stamp":"2024-03-13T08:11:16.030764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T10:09:57.248468+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":2498,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:24.729615+0000","last_change":"2024-03-13T08:11:24.730375+0000","last_active":"2024-03-13T08:11:24.729615+0000","last_peered":"2024-03-13T08:11:24.729615+0000","last_clean":"2024-03-13T08:11:24.729615+0000","last_became_active":"2024-03-13T07:36:13.088822+0000","last_became_peered":"2024-03-13T07:36:13.088822+0000","last_unstale":"2024-03-13T08:11:24.729615+0000","last_undegraded":"2024-03-13T08:11:24.729615+0000","last_fullsized":"2024-03-13T08:11:24.729615+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:24.728075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:24.728075+0000","last_clean_scrub_stamp":"2024-03-13T08:11:24.728075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T12:38:20.023090+0000","scrub_duration":70,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2503,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:19.509438+0000","last_change":"2024-03-13T08:11:19.510278+0000","last_active":"2024-03-13T08:11:19.509438+0000","last_peered":"2024-03-13T08:11:19.509438+0000","last_clean":"2024-03-13T08:11:19.509438+0000","last_became_active":"2024-03-13T07:34:52.014922+0000","last_became_peered":"2024-03-13T07:34:52.014922+0000","last_unstale":"2024-03-13T08:11:19.509438+0000","last_undegraded":"2024-03-13T08:11:19.509438+0000","last_fullsized":"2024-03-13T08:11:19.509438+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:19.501870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:19.501870+0000","last_clean_scrub_stamp":"2024-03-13T08:11:19.501870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:02:52.031057+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":2506,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:17.670059+0000","last_change":"2024-03-13T08:11:17.670789+0000","last_active":"2024-03-13T08:11:17.670059+0000","last_peered":"2024-03-13T08:11:17.670059+0000","last_clean":"2024-03-13T08:11:17.670059+0000","last_became_active":"2024-03-13T07:36:30.750203+0000","last_became_peered":"2024-03-13T07:36:30.750203+0000","last_unstale":"2024-03-13T08:11:17.670059+0000","last_undegraded":"2024-03-13T08:11:17.670059+0000","last_fullsized":"2024-03-13T08:11:17.670059+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:17.668541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:17.668541+0000","last_clean_scrub_stamp":"2024-03-13T08:11:17.668541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T09:22:47.800996+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":2522,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:20.081426+0000","last_change":"2024-03-13T08:11:20.082227+0000","last_active":"2024-03-13T08:11:20.081426+0000","last_peered":"2024-03-13T08:11:20.081426+0000","last_clean":"2024-03-13T08:11:20.081426+0000","last_became_active":"2024-03-13T07:34:52.909827+0000","last_became_peered":"2024-03-13T07:34:52.909827+0000","last_unstale":"2024-03-13T08:11:20.081426+0000","last_undegraded":"2024-03-13T08:11:20.081426+0000","last_fullsized":"2024-03-13T08:11:20.081426+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:20.079493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:20.079493+0000","last_clean_scrub_stamp":"2024-03-13T08:11:20.079493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T10:15:34.892089+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":2500,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:18.673040+0000","last_change":"2024-03-13T08:11:18.673779+0000","last_active":"2024-03-13T08:11:18.673040+0000","last_peered":"2024-03-13T08:11:18.673040+0000","last_clean":"2024-03-13T08:11:18.673040+0000","last_became_active":"2024-03-13T07:35:55.507183+0000","last_became_peered":"2024-03-13T07:35:55.507183+0000","last_unstale":"2024-03-13T08:11:18.673040+0000","last_undegraded":"2024-03-13T08:11:18.673040+0000","last_fullsized":"2024-03-13T08:11:18.673040+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:18.671514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:18.671514+0000","last_clean_scrub_stamp":"2024-03-13T08:11:18.671514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:52:49.477010+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2509,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:17.708474+0000","last_change":"2024-03-13T08:11:17.709258+0000","last_active":"2024-03-13T08:11:17.708474+0000","last_peered":"2024-03-13T08:11:17.708474+0000","last_clean":"2024-03-13T08:11:17.708474+0000","last_became_active":"2024-03-13T07:34:52.922427+0000","last_became_peered":"2024-03-13T07:34:52.922427+0000","last_unstale":"2024-03-13T08:11:17.708474+0000","last_undegraded":"2024-03-13T08:11:17.708474+0000","last_fullsized":"2024-03-13T08:11:17.708474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:17.706851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:17.706851+0000","last_clean_scrub_stamp":"2024-03-13T08:11:17.706851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T09:57:03.209671+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":2499,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:21.112984+0000","last_change":"2024-03-13T08:11:21.113714+0000","last_active":"2024-03-13T08:11:21.112984+0000","last_peered":"2024-03-13T08:11:21.112984+0000","last_clean":"2024-03-13T08:11:21.112984+0000","last_became_active":"2024-03-13T07:36:03.416513+0000","last_became_peered":"2024-03-13T07:36:03.416513+0000","last_unstale":"2024-03-13T08:11:21.112984+0000","last_undegraded":"2024-03-13T08:11:21.112984+0000","last_fullsized":"2024-03-13T08:11:21.112984+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:21.111465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:21.111465+0000","last_clean_scrub_stamp":"2024-03-13T08:11:21.111465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:12:40.864486+0000","scrub_duration":84,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2521,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:17.077078+0000","last_change":"2024-03-13T08:11:17.077868+0000","last_active":"2024-03-13T08:11:17.077078+0000","last_peered":"2024-03-13T08:11:17.077078+0000","last_clean":"2024-03-13T08:11:17.077078+0000","last_became_active":"2024-03-13T07:34:52.128201+0000","last_became_peered":"2024-03-13T07:34:52.128201+0000","last_unstale":"2024-03-13T08:11:17.077078+0000","last_undegraded":"2024-03-13T08:11:17.077078+0000","last_fullsized":"2024-03-13T08:11:17.077078+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:17.075534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:17.075534+0000","last_clean_scrub_stamp":"2024-03-13T08:11:17.075534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:57:40.094199+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":2499,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:18.098281+0000","last_change":"2024-03-13T08:11:18.099114+0000","last_active":"2024-03-13T08:11:18.098281+0000","last_peered":"2024-03-13T08:11:18.098281+0000","last_clean":"2024-03-13T08:11:18.098281+0000","last_became_active":"2024-03-13T07:35:58.686006+0000","last_became_peered":"2024-03-13T07:35:58.686006+0000","last_unstale":"2024-03-13T08:11:18.098281+0000","last_undegraded":"2024-03-13T08:11:18.098281+0000","last_fullsized":"2024-03-13T08:11:18.098281+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:18.096678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:18.096678+0000","last_clean_scrub_stamp":"2024-03-13T08:11:18.096678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:33:16.538414+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2507,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:17.385969+0000","last_change":"2024-03-13T08:11:17.386726+0000","last_active":"2024-03-13T08:11:17.385969+0000","last_peered":"2024-03-13T08:11:17.385969+0000","last_clean":"2024-03-13T08:11:17.385969+0000","last_became_active":"2024-03-13T07:34:52.404260+0000","last_became_peered":"2024-03-13T07:34:52.404260+0000","last_unstale":"2024-03-13T08:11:17.385969+0000","last_undegraded":"2024-03-13T08:11:17.385969+0000","last_fullsized":"2024-03-13T08:11:17.385969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:17.384452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:17.384452+0000","last_clean_scrub_stamp":"2024-03-13T08:11:17.384452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T19:35:02.381282+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":2511,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:23.123896+0000","last_change":"2024-03-13T08:11:23.124632+0000","last_active":"2024-03-13T08:11:23.123896+0000","last_peered":"2024-03-13T08:11:23.123896+0000","last_clean":"2024-03-13T08:11:23.123896+0000","last_became_active":"2024-03-13T07:36:24.325717+0000","last_became_peered":"2024-03-13T07:36:24.325717+0000","last_unstale":"2024-03-13T08:11:23.123896+0000","last_undegraded":"2024-03-13T08:11:23.123896+0000","last_fullsized":"2024-03-13T08:11:23.123896+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:23.122422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:23.122422+0000","last_clean_scrub_stamp":"2024-03-13T08:11:23.122422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:22:50.186929+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":2519,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:19.131512+0000","last_change":"2024-03-13T08:11:19.132301+0000","last_active":"2024-03-13T08:11:19.131512+0000","last_peered":"2024-03-13T08:11:19.131512+0000","last_clean":"2024-03-13T08:11:19.131512+0000","last_became_active":"2024-03-13T07:34:52.384430+0000","last_became_peered":"2024-03-13T07:34:52.384430+0000","last_unstale":"2024-03-13T08:11:19.131512+0000","last_undegraded":"2024-03-13T08:11:19.131512+0000","last_fullsized":"2024-03-13T08:11:19.131512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:19.129940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:19.129940+0000","last_clean_scrub_stamp":"2024-03-13T08:11:19.129940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:20:58.508185+0000","scrub_duration":50,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":2493,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:21.502211+0000","last_change":"2024-03-13T08:11:21.502963+0000","last_active":"2024-03-13T08:11:21.502211+0000","last_peered":"2024-03-13T08:11:21.502211+0000","last_clean":"2024-03-13T08:11:21.502211+0000","last_became_active":"2024-03-13T07:35:57.201526+0000","last_became_peered":"2024-03-13T07:35:57.201526+0000","last_unstale":"2024-03-13T08:11:21.502211+0000","last_undegraded":"2024-03-13T08:11:21.502211+0000","last_fullsized":"2024-03-13T08:11:21.502211+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:21.500626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:21.500626+0000","last_clean_scrub_stamp":"2024-03-13T08:11:21.500626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T13:55:57.168796+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":2509,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:22.506912+0000","last_change":"2024-03-13T08:11:22.507656+0000","last_active":"2024-03-13T08:11:22.506912+0000","last_peered":"2024-03-13T08:11:22.506912+0000","last_clean":"2024-03-13T08:11:22.506912+0000","last_became_active":"2024-03-13T07:36:33.040995+0000","last_became_peered":"2024-03-13T07:36:33.040995+0000","last_unstale":"2024-03-13T08:11:22.506912+0000","last_undegraded":"2024-03-13T08:11:22.506912+0000","last_fullsized":"2024-03-13T08:11:22.506912+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:22.505420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:22.505420+0000","last_clean_scrub_stamp":"2024-03-13T08:11:22.505420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:17:51.401656+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":2509,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:21.505224+0000","last_change":"2024-03-13T08:11:21.505966+0000","last_active":"2024-03-13T08:11:21.505224+0000","last_peered":"2024-03-13T08:11:21.505224+0000","last_clean":"2024-03-13T08:11:21.505224+0000","last_became_active":"2024-03-13T07:35:59.327404+0000","last_became_peered":"2024-03-13T07:35:59.327404+0000","last_unstale":"2024-03-13T08:11:21.505224+0000","last_undegraded":"2024-03-13T08:11:21.505224+0000","last_fullsized":"2024-03-13T08:11:21.505224+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:21.503615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:21.503615+0000","last_clean_scrub_stamp":"2024-03-13T08:11:21.503615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:46:51.595360+0000","scrub_duration":81,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":2498,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:21.718266+0000","last_change":"2024-03-13T08:11:21.719108+0000","last_active":"2024-03-13T08:11:21.718266+0000","last_peered":"2024-03-13T08:11:21.718266+0000","last_clean":"2024-03-13T08:11:21.718266+0000","last_became_active":"2024-03-13T07:35:58.453584+0000","last_became_peered":"2024-03-13T07:35:58.453584+0000","last_unstale":"2024-03-13T08:11:21.718266+0000","last_undegraded":"2024-03-13T08:11:21.718266+0000","last_fullsized":"2024-03-13T08:11:21.718266+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:21.716611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:21.716611+0000","last_clean_scrub_stamp":"2024-03-13T08:11:21.716611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:19:00.121603+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":2507,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:19.461950+0000","last_change":"2024-03-13T08:11:19.462715+0000","last_active":"2024-03-13T08:11:19.461950+0000","last_peered":"2024-03-13T08:11:19.461950+0000","last_clean":"2024-03-13T08:11:19.461950+0000","last_became_active":"2024-03-13T07:35:58.650418+0000","last_became_peered":"2024-03-13T07:35:58.650418+0000","last_unstale":"2024-03-13T08:11:19.461950+0000","last_undegraded":"2024-03-13T08:11:19.461950+0000","last_fullsized":"2024-03-13T08:11:19.461950+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:19.460387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:19.460387+0000","last_clean_scrub_stamp":"2024-03-13T08:11:19.460387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T11:05:57.403800+0000","scrub_duration":82,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":2505,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:16.699663+0000","last_change":"2024-03-13T08:11:16.700435+0000","last_active":"2024-03-13T08:11:16.699663+0000","last_peered":"2024-03-13T08:11:16.699663+0000","last_clean":"2024-03-13T08:11:16.699663+0000","last_became_active":"2024-03-13T07:35:58.196136+0000","last_became_peered":"2024-03-13T07:35:58.196136+0000","last_unstale":"2024-03-13T08:11:16.699663+0000","last_undegraded":"2024-03-13T08:11:16.699663+0000","last_fullsized":"2024-03-13T08:11:16.699663+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:16.698073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:16.698073+0000","last_clean_scrub_stamp":"2024-03-13T08:11:16.698073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T15:42:45.073969+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":2499,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:19.165854+0000","last_change":"2024-03-13T08:11:19.167126+0000","last_active":"2024-03-13T08:11:19.165854+0000","last_peered":"2024-03-13T08:11:19.165854+0000","last_clean":"2024-03-13T08:11:19.165854+0000","last_became_active":"2024-03-13T07:36:01.627175+0000","last_became_peered":"2024-03-13T07:36:01.627175+0000","last_unstale":"2024-03-13T08:11:19.165854+0000","last_undegraded":"2024-03-13T08:11:19.165854+0000","last_fullsized":"2024-03-13T08:11:19.165854+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:19.161504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:19.161504+0000","last_clean_scrub_stamp":"2024-03-13T08:11:19.161504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:49:14.162417+0000","scrub_duration":72,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":2505,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:15.731044+0000","last_change":"2024-03-13T08:11:15.731801+0000","last_active":"2024-03-13T08:11:15.731044+0000","last_peered":"2024-03-13T08:11:15.731044+0000","last_clean":"2024-03-13T08:11:15.731044+0000","last_became_active":"2024-03-13T07:35:57.866652+0000","last_became_peered":"2024-03-13T07:35:57.866652+0000","last_unstale":"2024-03-13T08:11:15.731044+0000","last_undegraded":"2024-03-13T08:11:15.731044+0000","last_fullsized":"2024-03-13T08:11:15.731044+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:15.729486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:15.729486+0000","last_clean_scrub_stamp":"2024-03-13T08:11:15.729486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T12:29:50.211586+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":2511,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:21.088645+0000","last_change":"2024-03-13T08:11:21.089391+0000","last_active":"2024-03-13T08:11:21.088645+0000","last_peered":"2024-03-13T08:11:21.088645+0000","last_clean":"2024-03-13T08:11:21.088645+0000","last_became_active":"2024-03-13T07:35:55.763372+0000","last_became_peered":"2024-03-13T07:35:55.763372+0000","last_unstale":"2024-03-13T08:11:21.088645+0000","last_undegraded":"2024-03-13T08:11:21.088645+0000","last_fullsized":"2024-03-13T08:11:21.088645+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:21.087130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:21.087130+0000","last_clean_scrub_stamp":"2024-03-13T08:11:21.087130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T14:29:38.813172+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":2526,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:14.605043+0000","last_change":"2024-03-13T08:11:14.605806+0000","last_active":"2024-03-13T08:11:14.605043+0000","last_peered":"2024-03-13T08:11:14.605043+0000","last_clean":"2024-03-13T08:11:14.605043+0000","last_became_active":"2024-03-13T07:35:58.764470+0000","last_became_peered":"2024-03-13T07:35:58.764470+0000","last_unstale":"2024-03-13T08:11:14.605043+0000","last_undegraded":"2024-03-13T08:11:14.605043+0000","last_fullsized":"2024-03-13T08:11:14.605043+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:14.603464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:14.603464+0000","last_clean_scrub_stamp":"2024-03-13T08:11:14.603464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T19:01:12.081699+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":2498,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:20.161246+0000","last_change":"2024-03-13T08:11:20.162540+0000","last_active":"2024-03-13T08:11:20.161246+0000","last_peered":"2024-03-13T08:11:20.161246+0000","last_clean":"2024-03-13T08:11:20.161246+0000","last_became_active":"2024-03-13T07:36:03.227060+0000","last_became_peered":"2024-03-13T07:36:03.227060+0000","last_unstale":"2024-03-13T08:11:20.161246+0000","last_undegraded":"2024-03-13T08:11:20.161246+0000","last_fullsized":"2024-03-13T08:11:20.161246+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:20.158475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:20.158475+0000","last_clean_scrub_stamp":"2024-03-13T08:11:20.158475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T09:43:00.223857+0000","scrub_duration":95,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":2499,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:22.721195+0000","last_change":"2024-03-13T08:11:22.721968+0000","last_active":"2024-03-13T08:11:22.721195+0000","last_peered":"2024-03-13T08:11:22.721195+0000","last_clean":"2024-03-13T08:11:22.721195+0000","last_became_active":"2024-03-13T07:35:58.461765+0000","last_became_peered":"2024-03-13T07:35:58.461765+0000","last_unstale":"2024-03-13T08:11:22.721195+0000","last_undegraded":"2024-03-13T08:11:22.721195+0000","last_fullsized":"2024-03-13T08:11:22.721195+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:22.719624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:22.719624+0000","last_clean_scrub_stamp":"2024-03-13T08:11:22.719624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T18:30:48.160682+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":2510,"reported_epoch":1293,"state":"active+clean","last_fresh":"2024-03-13T08:11:22.068767+0000","last_change":"2024-03-13T08:11:22.069503+0000","last_active":"2024-03-13T08:11:22.068767+0000","last_peered":"2024-03-13T08:11:22.068767+0000","last_clean":"2024-03-13T08:11:22.068767+0000","last_became_active":"2024-03-13T07:36:14.238576+0000","last_became_peered":"2024-03-13T07:36:14.238576+0000","last_unstale":"2024-03-13T08:11:22.068767+0000","last_undegraded":"2024-03-13T08:11:22.068767+0000","last_fullsized":"2024-03-13T08:11:22.068767+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T08:11:22.067210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T08:11:22.067210+0000","last_clean_scrub_stamp":"2024-03-13T08:11:22.067210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T12:50:28.190146+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":20,"seq":85899346363,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72096,"kb_used_data":4824,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299744,"statfs":{"total":96636764160,"available":96562937856,"internally_reserved":0,"allocated":4939776,"data_stored":2245998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":20,"seq":85899346364,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72096,"kb_used_data":4824,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299744,"statfs":{"total":96636764160,"available":96562937856,"internally_reserved":0,"allocated":4939776,"data_stored":2245998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":20,"seq":85899346365,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72488,"kb_used_data":5216,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299352,"statfs":{"total":96636764160,"available":96562536448,"internally_reserved":0,"allocated":5341184,"data_stored":2643838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":18,"seq":77309411777,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72100,"kb_used_data":4828,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299740,"statfs":{"total":96636764160,"available":96562933760,"internally_reserved":0,"allocated":4943872,"data_stored":2246003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":16,"seq":68719477179,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72096,"kb_used_data":4824,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299744,"statfs":{"total":96636764160,"available":96562937856,"internally_reserved":0,"allocated":4939776,"data_stored":2245998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509884,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72096,"kb_used_data":4824,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299744,"statfs":{"total":96636764160,"available":96562937856,"internally_reserved":0,"allocated":4939776,"data_stored":2245998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834575291,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72100,"kb_used_data":4828,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94299740,"statfs":{"total":96636764160,"available":96562933760,"internally_reserved":0,"allocated":4943872,"data_stored":2246003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":68876747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575292,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76648,"kb_used_data":5216,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295192,"statfs":{"total":96636764160,"available":96558276608,"internally_reserved":0,"allocated":5341184,"data_stored":2643838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-13T08:11:40.118 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-13T08:11:40.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.219+0000 7f650de2e640 10 monclient: get_monmap_and_config 2024-03-13T08:11:40.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.219+0000 7f650de2e640 10 monclient: build_initial_monmap 2024-03-13T08:11:40.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.219+0000 7f650de2e640 10 monclient: monmap: 2024-03-13T08:11:40.220 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:40.220 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:40.220 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T08:11:40.220309+0000 2024-03-13T08:11:40.220 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T08:11:40.220309+0000 2024-03-13T08:11:40.220 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:40.220 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:40.220 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.132:6789/0 mon.noname-a-legacy 2024-03-13T08:11:40.220 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.195:6789/0 mon.noname-c-legacy 2024-03-13T08:11:40.220 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-b 2024-03-13T08:11:40.220 INFO:teuthology.orchestra.run.smithi132.stderr:3: v2:172.21.15.132:3300/0 mon.noname-a 2024-03-13T08:11:40.220 INFO:teuthology.orchestra.run.smithi132.stderr:4: v2:172.21.15.195:3300/0 mon.noname-c 2024-03-13T08:11:40.221 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:40.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.220+0000 7f650de2e640 1 Processor -- start 2024-03-13T08:11:40.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.220+0000 7f650de2e640 1 -- start start 2024-03-13T08:11:40.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.220+0000 7f650de2e640 10 monclient: init 2024-03-13T08:11:40.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.220+0000 7f650de2e640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:40.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.220+0000 7f650de2e640 10 monclient: _add_conns ranks=[3,1,0,2,4] 2024-03-13T08:11:40.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.220+0000 7f650de2e640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f65080b5310 0x7f650813f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:40.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.220+0000 7f650de2e640 10 monclient(hunting): picked mon.noname-a con 0x7f65080b5310 addr v2:172.21.15.132:3300/0 2024-03-13T08:11:40.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.220+0000 7f650de2e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f65080afa20 addr v1:172.21.15.195:6789/0 2024-03-13T08:11:40.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.220+0000 7f650de2e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f650813fea0 addr v1:172.21.15.132:6789/0 2024-03-13T08:11:40.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.220+0000 7f650de2e640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65081402b0 con 0x7f650813fea0 2024-03-13T08:11:40.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.220+0000 7f650de2e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65080b16e0 con 0x7f65080afa20 2024-03-13T08:11:40.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.220+0000 7f650de2e640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:40.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.220+0000 7f650de2e640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65080b56f0 con 0x7f65080b5310 2024-03-13T08:11:40.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.220+0000 7f650de2e640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:40.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.220+0000 7f650de2e640 10 monclient(hunting): authenticate will time out at 3527.629023s 2024-03-13T08:11:40.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.221+0000 7f6507fff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f650813fea0 0x7f650814cf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:35828/0 (socket says 172.21.15.132:35828) 2024-03-13T08:11:40.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.221+0000 7f6507fff640 1 -- 172.21.15.132:0/1898594110 learned_addr learned my addr 172.21.15.132:0/1898594110 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-03-13T08:11:40.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.221+0000 7f65077fe640 1 --2- 172.21.15.132:0/1898594110 >> v2:172.21.15.132:3300/0 conn(0x7f65080b5310 0x7f650813f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:40.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.222+0000 7f65077fe640 10 monclient(hunting): get_auth_request con 0x7f65080b5310 auth_method 0 2024-03-13T08:11:40.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.222+0000 7f65077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:40.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.222+0000 7f65077fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:40.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.222+0000 7f65077fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:40.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.223+0000 7f65077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:40.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.223+0000 7f65077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:40.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.223+0000 7f65077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:40.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.224+0000 7f65067fc640 1 -- 172.21.15.132:0/1898594110 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583423275 0 0) 0x7f65080b16e0 con 0x7f65080afa20 2024-03-13T08:11:40.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.224+0000 7f65067fc640 10 monclient(hunting): _negotiate 2024-03-13T08:11:40.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.224+0000 7f65067fc640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:40.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.224+0000 7f65067fc640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:40.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.224+0000 7f65067fc640 10 monclient(hunting): my global_id is 19753 2024-03-13T08:11:40.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.224+0000 7f65067fc640 1 -- 172.21.15.132:0/1898594110 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64f0003520 con 0x7f65080afa20 2024-03-13T08:11:40.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.224+0000 7f65077fe640 10 monclient(hunting): handle_auth_done global_id 16110 payload 274 2024-03-13T08:11:40.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.224+0000 7f65077fe640 1 -- 172.21.15.132:0/1898594110 >> v1:172.21.15.195:6789/0 conn(0x7f65080afa20 legacy=0x7f65080b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:40.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.224+0000 7f65077fe640 1 -- 172.21.15.132:0/1898594110 >> v1:172.21.15.132:6789/0 conn(0x7f650813fea0 legacy=0x7f650814cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:40.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.224+0000 7f65077fe640 10 monclient: _finish_hunting 0 2024-03-13T08:11:40.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.224+0000 7f65077fe640 1 monclient: found mon.noname-a 2024-03-13T08:11:40.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.224+0000 7f65077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:40.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.224+0000 7f65077fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T08:11:40.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.224+0000 7f65077fe640 1 -- 172.21.15.132:0/1898594110 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65080afeb0 con 0x7f65080b5310 2024-03-13T08:11:40.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.228+0000 7f65077fe640 1 --2- 172.21.15.132:0/1898594110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65080b5310 0x7f650813f960 secure :-1 s=READY pgs=3280 cs=0 l=1 rev1=1 crypto rx=0x7f64f80097a0 tx=0x7f64f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee7486e28ec9154 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:40.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.232+0000 7f65067fc640 1 -- 172.21.15.132:0/1898594110 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f80315a0 con 0x7f65080b5310 2024-03-13T08:11:40.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.232+0000 7f65067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:40.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.232+0000 7f65067fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T08:11:40.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.232+0000 7f65067fc640 10 monclient: dump: 2024-03-13T08:11:40.233 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:40.233 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:40.233 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:40.233 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:40.233 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:40.234 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:40.234 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:40.234 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:40.234 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:40.234 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:40.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.232+0000 7f65067fc640 10 monclient: _finish_auth 0 2024-03-13T08:11:40.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.232+0000 7f65067fc640 10 monclient: _check_auth_tickets 2024-03-13T08:11:40.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.232+0000 7f65067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:40.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.232+0000 7f65067fc640 1 -- 172.21.15.132:0/1898594110 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f8037070 con 0x7f65080b5310 2024-03-13T08:11:40.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.232+0000 7f65067fc640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:40.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.232+0000 7f65067fc640 1 -- 172.21.15.132:0/1898594110 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8031960 con 0x7f65080b5310 2024-03-13T08:11:40.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.232+0000 7f650de2e640 5 monclient: authenticate success, global_id 16110 2024-03-13T08:11:40.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.232+0000 7f650de2e640 10 monclient: get_monmap_and_config success 2024-03-13T08:11:40.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.232+0000 7f650de2e640 4 set_mon_vals no callback set 2024-03-13T08:11:40.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.232+0000 7f65067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:40.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.232+0000 7f65067fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T08:11:40.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.232+0000 7f65067fc640 10 monclient: dump: 2024-03-13T08:11:40.235 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:40.235 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:40.235 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:40.235 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:40.235 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:40.235 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:40.235 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:40.235 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:40.235 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:40.235 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:40.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.233+0000 7f650de2e640 10 monclient: shutdown 2024-03-13T08:11:40.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.233+0000 7f650de2e640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:40.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.233+0000 7f650de2e640 1 -- 172.21.15.132:0/1898594110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65080b5310 msgr2=0x7f650813f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:40.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.233+0000 7f650de2e640 1 --2- 172.21.15.132:0/1898594110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65080b5310 0x7f650813f960 secure :-1 s=READY pgs=3280 cs=0 l=1 rev1=1 crypto rx=0x7f64f80097a0 tx=0x7f64f802b840 comp rx=0 tx=0).stop 2024-03-13T08:11:40.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.233+0000 7f650de2e640 1 -- 172.21.15.132:0/1898594110 shutdown_connections 2024-03-13T08:11:40.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.233+0000 7f650de2e640 1 --2- 172.21.15.132:0/1898594110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65080b5310 0x7f650813f960 unknown :-1 s=CLOSED pgs=3280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:40.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.233+0000 7f650de2e640 1 -- 172.21.15.132:0/1898594110 >> 172.21.15.132:0/1898594110 conn(0x7f650806d2b0 msgr2=0x7f650806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:40.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.233+0000 7f650de2e640 1 -- 172.21.15.132:0/1898594110 shutdown_connections 2024-03-13T08:11:40.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.233+0000 7f650de2e640 1 -- 172.21.15.132:0/1898594110 wait complete. 2024-03-13T08:11:40.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.234+0000 7f650de2e640 10 monclient: build_initial_monmap 2024-03-13T08:11:40.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.234+0000 7f650de2e640 10 monclient: monmap: 2024-03-13T08:11:40.240 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-03-13T08:11:40.240 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-13T08:11:40.240 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-03-13T08:11:40.240 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-03-13T08:11:40.240 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-03-13T08:11:40.240 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:40.240 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.noname-a 2024-03-13T08:11:40.240 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-03-13T08:11:40.240 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.noname-c 2024-03-13T08:11:40.240 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:40.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.234+0000 7f650de2e640 1 librados: starting msgr at 2024-03-13T08:11:40.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.234+0000 7f650de2e640 1 librados: starting objecter 2024-03-13T08:11:40.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 1 Processor -- start 2024-03-13T08:11:40.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 1 -- start start 2024-03-13T08:11:40.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 1 librados: setting wanted keys 2024-03-13T08:11:40.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 1 librados: calling monclient init 2024-03-13T08:11:40.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 10 monclient: init 2024-03-13T08:11:40.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 10 monclient: _reopen_session rank -1 2024-03-13T08:11:40.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 10 monclient: _add_conns ranks=[0,1,2] 2024-03-13T08:11:40.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65080b5310 0x7f65080b8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:40.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 10 monclient(hunting): picked mon.noname-a con 0x7f65080b5310 addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-03-13T08:11:40.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f650813fea0 0x7f65080b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:40.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 10 monclient(hunting): picked mon.noname-b con 0x7f650813fea0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-03-13T08:11:40.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65080b8a40 0x7f65080b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:40.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 10 monclient(hunting): picked mon.noname-c con 0x7f65080b8a40 addr [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] 2024-03-13T08:11:40.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:40.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65080b0550 con 0x7f65080b5310 2024-03-13T08:11:40.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:40.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65080b03d0 con 0x7f650813fea0 2024-03-13T08:11:40.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 10 monclient(hunting): start opening mon connection 2024-03-13T08:11:40.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65080b2210 con 0x7f65080b8a40 2024-03-13T08:11:40.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 10 monclient(hunting): _renew_subs 2024-03-13T08:11:40.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.235+0000 7f650de2e640 10 monclient(hunting): authenticate will time out at 3527.643879s 2024-03-13T08:11:40.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.236+0000 7f65077fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65080b5310 0x7f65080b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:40.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.236+0000 7f6507fff640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65080b8a40 0x7f65080b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:40.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.236+0000 7f65077fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65080b5310 0x7f65080b8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36974/0 (socket says 172.21.15.132:36974) 2024-03-13T08:11:40.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.236+0000 7f65077fe640 1 -- 172.21.15.132:0/1391760833 learned_addr learned my addr 172.21.15.132:0/1391760833 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-13T08:11:40.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.236+0000 7f65077fe640 10 monclient(hunting): get_auth_request con 0x7f65080b5310 auth_method 0 2024-03-13T08:11:40.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.236+0000 7f65077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:40.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.236+0000 7f65077fe640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:40.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.236+0000 7f65077fe640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:40.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.236+0000 7f6506ffd640 1 --2- 172.21.15.132:0/1391760833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f650813fea0 0x7f65080b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:40.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.236+0000 7f6507fff640 10 monclient(hunting): get_auth_request con 0x7f65080b8a40 auth_method 0 2024-03-13T08:11:40.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.236+0000 7f6507fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:40.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.236+0000 7f6507fff640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:40.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.236+0000 7f6507fff640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:40.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.236+0000 7f65077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:40.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.236+0000 7f65077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:40.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.236+0000 7f65077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:40.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.237+0000 7f6506ffd640 10 monclient(hunting): get_auth_request con 0x7f650813fea0 auth_method 0 2024-03-13T08:11:40.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.237+0000 7f6506ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-13T08:11:40.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.237+0000 7f6506ffd640 10 monclient(hunting): _init_auth method 2 2024-03-13T08:11:40.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.237+0000 7f6506ffd640 10 monclient(hunting): _init_auth creating new auth 2024-03-13T08:11:40.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.237+0000 7f6507fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:40.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.237+0000 7f6507fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:40.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.237+0000 7f6507fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:40.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.237+0000 7f6506ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-13T08:11:40.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.237+0000 7f6506ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-13T08:11:40.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.237+0000 7f6506ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-13T08:11:40.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.239+0000 7f65077fe640 10 monclient(hunting): handle_auth_done global_id 16116 payload 824 2024-03-13T08:11:40.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.239+0000 7f65077fe640 1 -- 172.21.15.132:0/1391760833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65080b8a40 msgr2=0x7f65080b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:40.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.239+0000 7f65077fe640 1 --2- 172.21.15.132:0/1391760833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65080b8a40 0x7f65080b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:40.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.239+0000 7f65077fe640 1 -- 172.21.15.132:0/1391760833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f650813fea0 msgr2=0x7f65080b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:40.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.239+0000 7f65077fe640 1 --2- 172.21.15.132:0/1391760833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f650813fea0 0x7f65080b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:40.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.239+0000 7f65077fe640 10 monclient: _finish_hunting 0 2024-03-13T08:11:40.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.239+0000 7f65077fe640 1 monclient: found mon.noname-a 2024-03-13T08:11:40.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.239+0000 7f65077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-03-13T08:11:40.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.239+0000 7f65077fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.132:3300/0 2024-03-13T08:11:40.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.239+0000 7f65077fe640 1 -- 172.21.15.132:0/1391760833 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65080b7820 con 0x7f65080b5310 2024-03-13T08:11:40.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.242+0000 7f65077fe640 1 --2- 172.21.15.132:0/1391760833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65080b5310 0x7f65080b8500 secure :-1 s=READY pgs=3281 cs=0 l=1 rev1=1 crypto rx=0x7f64f8034040 tx=0x7f64f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:40.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.242+0000 7f64effff640 10 client.?.objecter ms_handle_connect 0x7f65080b5310 2024-03-13T08:11:40.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.242+0000 7f64effff640 10 client.?.objecter resend_mon_ops 2024-03-13T08:11:40.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.244+0000 7f64effff640 1 -- 172.21.15.132:0/1391760833 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8032840 con 0x7f65080b5310 2024-03-13T08:11:40.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.244+0000 7f64effff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:40.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.244+0000 7f64effff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-13T08:11:40.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.244+0000 7f64effff640 10 monclient: dump: 2024-03-13T08:11:40.248 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:40.248 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:40.248 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:40.248 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:40.248 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:40.248 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:40.248 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:40.248 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:40.249 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:40.249 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:40.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.244+0000 7f64effff640 10 monclient: _finish_auth 0 2024-03-13T08:11:40.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.244+0000 7f64effff640 10 monclient: _check_auth_tickets 2024-03-13T08:11:40.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.244+0000 7f64effff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-03-13T08:11:40.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.244+0000 7f650de2e640 5 monclient: authenticate success, global_id 16116 2024-03-13T08:11:40.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.244+0000 7f650de2e640 10 monclient: _renew_subs 2024-03-13T08:11:40.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.244+0000 7f650de2e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T08:11:40.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.244+0000 7f650de2e640 1 -- 172.21.15.132:0/1391760833 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65080b7a90 con 0x7f65080b5310 2024-03-13T08:11:40.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.244+0000 7f650de2e640 10 client.16116.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-03-13T08:11:40.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.244+0000 7f650de2e640 10 monclient: _renew_subs 2024-03-13T08:11:40.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.244+0000 7f650de2e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T08:11:40.249 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.244+0000 7f650de2e640 1 -- 172.21.15.132:0/1391760833 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6508136e30 con 0x7f65080b5310 2024-03-13T08:11:40.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.244+0000 7f650de2e640 1 librados: init done 2024-03-13T08:11:40.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.245+0000 7f650de2e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:40.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.245+0000 7f650de2e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:40.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.245+0000 7f650de2e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T08:11:40.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.245+0000 7f650de2e640 1 -- 172.21.15.132:0/1391760833 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650804f000 con 0x7f65080b5310 2024-03-13T08:11:40.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.245+0000 7f64effff640 1 -- 172.21.15.132:0/1391760833 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f8037070 con 0x7f65080b5310 2024-03-13T08:11:40.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.245+0000 7f64effff640 10 monclient: handle_config config(0 keys) v1 2024-03-13T08:11:40.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.245+0000 7f64effff640 1 -- 172.21.15.132:0/1391760833 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8031e90 con 0x7f65080b5310 2024-03-13T08:11:40.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.245+0000 7f64effff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-13T08:11:40.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.245+0000 7f64effff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-13T08:11:40.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.245+0000 7f64effff640 10 monclient: dump: 2024-03-13T08:11:40.250 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-03-13T08:11:40.251 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 74e4bc2f-6a5e-4423-a8c8-807970a2f96c 2024-03-13T08:11:40.251 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:40.251 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-03-13T07:33:05.110387+0000 2024-03-13T08:11:40.251 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-03-13T08:11:40.251 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-03-13T08:11:40.251 INFO:teuthology.orchestra.run.smithi132.stderr:0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-13T08:11:40.251 INFO:teuthology.orchestra.run.smithi132.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-13T08:11:40.251 INFO:teuthology.orchestra.run.smithi132.stderr:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-03-13T08:11:40.251 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:11:40.251 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.245+0000 7f65057fa640 4 set_mon_vals no callback set 2024-03-13T08:11:40.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.494+0000 7f64effff640 1 -- 172.21.15.132:0/1391760833 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 90136+0+0 (secure 0 0 0) 0x7f64f80329e0 con 0x7f65080b5310 2024-03-13T08:11:40.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.494+0000 7f64effff640 1 --2- 172.21.15.132:0/1391760833 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f64f406d590 0x7f64f406fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T08:11:40.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.494+0000 7f6506ffd640 1 --2- 172.21.15.132:0/1391760833 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f64f406d590 0x7f64f406fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T08:11:40.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.494+0000 7f6506ffd640 10 monclient: get_auth_request con 0x7f64f406d590 auth_method 0 2024-03-13T08:11:40.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.495+0000 7f64effff640 10 client.16116.objecter ms_handle_connect 0x7f64f406d590 2024-03-13T08:11:40.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.495+0000 7f6506ffd640 1 --2- 172.21.15.132:0/1391760833 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f64f406d590 0x7f64f406fa50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f64fc007fd0 tx=0x7f64fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T08:11:40.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.508+0000 7f64effff640 1 -- 172.21.15.132:0/1391760833 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1293..1293 src has 1008..1293) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f64f8078cc0 con 0x7f65080b5310 2024-03-13T08:11:40.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.508+0000 7f64effff640 10 client.16116.objecter ms_dispatch 0x7f6508000b90 osd_map(1293..1293 src has 1008..1293) v4 2024-03-13T08:11:40.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.508+0000 7f64effff640 3 client.16116.objecter handle_osd_map got epochs [1293,1293] > 0 2024-03-13T08:11:40.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.508+0000 7f64effff640 3 client.16116.objecter handle_osd_map decoding full epoch 1293 2024-03-13T08:11:40.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.508+0000 7f64effff640 20 client.16116.objecter dump_active .. 0 homeless 2024-03-13T08:11:40.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.508+0000 7f64effff640 1 -- 172.21.15.132:0/1391760833 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183657 (secure 0 0 0) 0x7f64f8040070 con 0x7f65080b5310 2024-03-13T08:11:40.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.508+0000 7f64effff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-03-13T08:11:40.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.508+0000 7f64effff640 10 monclient: _finish_command 1 = system:0 2024-03-13T08:11:40.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.690+0000 7f650de2e640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-03-13T08:11:40.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.690+0000 7f650de2e640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-03-13T08:11:40.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.690+0000 7f650de2e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.132:3300/0 2024-03-13T08:11:40.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.690+0000 7f650de2e640 1 -- 172.21.15.132:0/1391760833 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f650813e030 con 0x7f65080b5310 2024-03-13T08:11:40.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.772+0000 7f64effff640 1 -- 172.21.15.132:0/1391760833 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) v1 ==== 123+0+0 (secure 0 0 0) 0x7f64f807ac20 con 0x7f65080b5310 2024-03-13T08:11:40.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.773+0000 7f64effff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-03-13T08:11:40.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.773+0000 7f64effff640 10 monclient: _finish_command 2 = system:0 2024-03-13T08:11:40.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.773+0000 7f64effff640 1 -- 172.21.15.132:0/1391760833 <== mon.0 v2:172.21.15.132:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f64f807ecc0 con 0x7f65080b5310 2024-03-13T08:11:40.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.773+0000 7f64effff640 10 monclient: handle_config config(1 keys) v1 2024-03-13T08:11:40.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.773+0000 7f6505ffb640 4 set_mon_vals no callback set 2024-03-13T08:11:40.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.773+0000 7f6505ffb640 10 set_mon_vals mon_health_to_clog = false 2024-03-13T08:11:40.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 10 librados: watch_flush enter 2024-03-13T08:11:40.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 10 librados: watch_flush exit 2024-03-13T08:11:40.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 20 client.16116.objecter shutdown clearing up homeless session... 2024-03-13T08:11:40.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 10 client.16116.objecter successfully canceled tick 2024-03-13T08:11:40.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 1 -- 172.21.15.132:0/1391760833 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f64f406d590 msgr2=0x7f64f406fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:40.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 1 --2- 172.21.15.132:0/1391760833 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f64f406d590 0x7f64f406fa50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f64fc007fd0 tx=0x7f64fc009040 comp rx=0 tx=0).stop 2024-03-13T08:11:40.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 10 monclient: shutdown 2024-03-13T08:11:40.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 20 monclient: shutdown discarding 0 pending message(s) 2024-03-13T08:11:40.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 1 -- 172.21.15.132:0/1391760833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65080b5310 msgr2=0x7f65080b8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T08:11:40.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 1 --2- 172.21.15.132:0/1391760833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65080b5310 0x7f65080b8500 secure :-1 s=READY pgs=3281 cs=0 l=1 rev1=1 crypto rx=0x7f64f8034040 tx=0x7f64f80046c0 comp rx=0 tx=0).stop 2024-03-13T08:11:40.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 1 -- 172.21.15.132:0/1391760833 shutdown_connections 2024-03-13T08:11:40.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 1 --2- 172.21.15.132:0/1391760833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65080b8a40 0x7f65080b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:40.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 1 --2- 172.21.15.132:0/1391760833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f650813fea0 0x7f65080b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:40.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 1 --2- 172.21.15.132:0/1391760833 >> [v2:172.21.15.132:6800/3847550149,v1:172.21.15.132:6801/3847550149] conn(0x7f64f406d590 0x7f64f406fa50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:40.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 1 --2- 172.21.15.132:0/1391760833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65080b5310 0x7f65080b8500 unknown :-1 s=CLOSED pgs=3281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T08:11:40.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 1 -- 172.21.15.132:0/1391760833 >> 172.21.15.132:0/1391760833 conn(0x7f650806d2b0 msgr2=0x7f650813d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T08:11:40.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 1 -- 172.21.15.132:0/1391760833 shutdown_connections 2024-03-13T08:11:40.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 1 -- 172.21.15.132:0/1391760833 wait complete. 2024-03-13T08:11:40.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-13T08:11:40.775+0000 7f650de2e640 1 librados: shutdown 2024-03-13T08:11:40.789 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-13T08:11:40.790 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-13T08:11:40.790 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-13T08:11:40.790 INFO:teuthology.orchestra.run:waiting for 300 2024-03-13T08:11:40.803 INFO:tasks.ceph.osd.0.smithi132.stderr:2024-03-13T08:11:40.794+0000 ac79640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 35551) UID: 0 2024-03-13T08:11:40.805 INFO:tasks.ceph.osd.0.smithi132.stderr:2024-03-13T08:11:40.795+0000 ac79640 -1 osd.0 1293 *** Got signal Terminated *** 2024-03-13T08:11:45.134 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:09.024 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:45.184 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:09.054 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:45.184 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:09.074 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:45.185 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:09.075 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:45.187 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:09.057 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:45.189 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:09.079 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:45.202 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:09.072 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:45.203 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:09.093 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:45.206 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:09.096 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:46.892 INFO:tasks.ceph.osd.0:Stopped 2024-03-13T08:11:46.892 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-13T08:11:46.892 INFO:teuthology.orchestra.run:waiting for 300 2024-03-13T08:11:46.905 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T08:11:46.896+0000 ac79640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 35557) UID: 0 2024-03-13T08:11:46.905 INFO:tasks.ceph.osd.1.smithi132.stderr:2024-03-13T08:11:46.897+0000 ac79640 -1 osd.1 1295 *** Got signal Terminated *** 2024-03-13T08:11:47.869 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-03-13T08:11:50.194 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:14.084 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:50.197 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:14.087 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:50.200 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:14.070 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:50.204 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:14.074 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:50.210 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:14.100 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:50.215 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:14.085 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:50.244 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:14.134 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:50.247 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:14.136 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:50.259 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:14.147 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:11:54.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-03-13T08:12:05.198 INFO:tasks.ceph.osd.1:Stopped 2024-03-13T08:12:05.198 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-13T08:12:05.198 INFO:teuthology.orchestra.run:waiting for 300 2024-03-13T08:12:05.211 INFO:tasks.ceph.osd.2.smithi132.stderr:2024-03-13T08:12:05.201+0000 ac79640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 35565) UID: 0 2024-03-13T08:12:05.211 INFO:tasks.ceph.osd.2.smithi132.stderr:2024-03-13T08:12:05.202+0000 ac79640 -1 osd.2 1297 *** Got signal Terminated *** 2024-03-13T08:12:05.234 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:29.123 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:05.236 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:29.125 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:05.251 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:29.141 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:05.285 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:29.155 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:05.285 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:29.175 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:05.286 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:29.176 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:05.287 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:29.156 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:05.298 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:29.169 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:05.299 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:29.189 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:10.248 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:34.138 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:10.250 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:34.140 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:10.262 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:34.151 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:10.291 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:34.162 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:10.293 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:34.163 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:10.306 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:34.174 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:10.306 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:34.196 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:10.311 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:34.197 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:10.319 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:34.209 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:11.300 INFO:tasks.ceph.osd.2:Stopped 2024-03-13T08:12:11.301 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-13T08:12:11.301 INFO:teuthology.orchestra.run:waiting for 300 2024-03-13T08:12:11.313 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-03-13T08:12:11.305+0000 ac79640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 35571) UID: 0 2024-03-13T08:12:11.316 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-03-13T08:12:11.306+0000 ac79640 -1 osd.3 1299 *** Got signal Terminated *** 2024-03-13T08:12:25.281 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:49.169 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:25.282 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:49.172 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:25.297 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:49.187 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:25.330 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:49.200 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:25.332 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:49.222 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:25.333 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:49.203 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:25.333 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:49.223 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:25.348 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:49.218 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:25.349 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:49.239 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:30.301 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:54.191 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:30.302 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:54.192 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:30.314 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:38:54.201 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:30.345 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:54.215 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:30.348 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:54.219 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:30.354 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:54.242 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:30.354 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:54.244 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:30.359 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:38:54.229 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:30.363 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:38:54.253 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:45.329 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:09.219 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:45.333 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:09.223 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:45.345 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:09.234 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:45.376 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:09.265 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:45.379 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:09.269 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:45.382 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:09.252 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:45.383 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:09.254 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:45.391 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:09.281 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:45.398 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:09.268 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:50.344 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:14.233 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:50.347 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:14.237 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:50.357 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:14.247 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:50.395 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:14.264 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:50.396 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:14.266 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:50.410 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:14.276 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:50.412 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:14.302 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:50.416 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:14.305 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:12:50.426 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:14.315 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:05.329 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:29.219 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:05.381 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:29.270 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:05.393 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:29.283 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:05.428 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:29.317 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:05.429 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:29.319 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:05.432 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:29.302 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:05.434 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:29.304 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:05.442 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:29.332 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:05.447 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:29.318 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:10.345 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:34.235 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:10.394 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:34.284 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:10.403 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:34.293 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:10.435 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:34.325 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:10.441 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:34.330 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:10.444 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:34.314 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:10.445 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:34.315 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:10.450 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:34.340 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:10.455 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:34.325 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:25.432 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:49.322 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:25.435 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:49.325 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:25.455 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:49.345 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:25.484 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:49.354 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:25.487 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:49.357 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:25.489 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:49.378 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:25.490 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:49.380 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:25.501 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:49.371 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:25.511 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:49.400 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:30.441 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:54.330 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:30.447 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:54.337 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:30.461 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:39:54.351 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:30.490 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:54.380 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:30.494 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:54.383 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:30.497 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:54.367 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:30.498 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:54.369 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:30.503 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:39:54.393 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:30.509 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:39:54.379 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:45.482 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:09.372 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:45.487 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:09.376 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:45.506 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:09.396 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:45.534 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:09.424 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:45.535 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:09.425 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:45.537 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:09.407 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:45.538 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:09.409 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:45.550 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:09.439 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:45.552 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:09.422 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:50.498 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:14.388 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:50.504 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:14.391 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:50.515 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:14.405 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:50.547 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:14.417 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:50.548 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:14.419 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:50.552 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:14.442 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:50.554 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:14.443 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:50.559 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:14.429 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:13:50.564 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:14.453 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:05.549 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:29.439 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:05.553 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:29.443 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:05.567 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:29.456 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:05.600 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:29.489 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:05.600 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:29.470 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:05.601 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:29.490 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:05.602 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:29.472 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:05.614 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:29.504 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:05.616 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:29.486 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:08.719 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:32.608 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:08.721 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:32.611 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:08.742 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:32.632 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:08.773 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:32.642 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:08.773 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:32.644 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:08.783 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:32.673 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:08.784 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:32.674 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:08.787 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:32.657 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:08.797 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:32.687 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:10.562 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:34.432 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:10.563 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:34.433 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:10.563 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:34.453 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:10.569 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:34.458 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:10.573 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:34.443 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:10.582 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:34.471 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:10.614 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:34.504 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:10.619 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:34.509 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:10.628 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:34.518 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:25.706 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:49.595 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:25.759 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:49.649 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:25.790 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:49.680 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:25.808 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:49.678 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:25.809 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:49.680 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:25.815 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:49.704 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:25.816 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:49.706 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:25.828 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:49.697 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:25.835 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:49.724 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:30.685 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:54.575 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:30.688 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:54.578 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:30.700 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:40:54.589 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:30.734 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:54.604 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:30.735 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:54.606 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:30.737 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:54.627 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:30.739 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:54.628 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:30.746 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:40:54.616 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:30.749 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:40:54.639 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:45.676 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:09.566 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:45.726 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:09.616 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:45.739 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:09.629 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:45.773 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:09.662 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:45.777 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:09.666 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:45.778 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:09.648 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:45.780 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:09.650 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:45.791 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:09.681 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:45.794 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:09.664 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:50.733 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:14.622 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:50.737 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:14.627 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:50.751 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:14.641 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:50.786 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:14.676 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:50.788 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:14.658 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:50.789 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:14.679 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:50.790 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:14.660 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:50.799 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:14.689 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:50.800 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:14.670 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:14:56.047 INFO:tasks.ceph.osd.3:Stopped 2024-03-13T08:14:56.047 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-13T08:14:56.047 INFO:teuthology.orchestra.run:waiting for 300 2024-03-13T08:14:56.064 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T08:14:56.052+0000 ac79640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 35241) UID: 0 2024-03-13T08:14:56.065 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-13T08:14:56.053+0000 ac79640 -1 osd.4 1301 *** Got signal Terminated *** 2024-03-13T08:15:00.458 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-03-13T08:15:02.149 INFO:tasks.ceph.osd.4:Stopped 2024-03-13T08:15:02.149 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-13T08:15:02.150 INFO:teuthology.orchestra.run:waiting for 300 2024-03-13T08:15:02.164 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-03-13T08:15:02.154+0000 ac79640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 35242) UID: 0 2024-03-13T08:15:02.164 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-03-13T08:15:02.155+0000 ac79640 -1 osd.5 1303 *** Got signal Terminated *** 2024-03-13T08:15:05.796 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:29.685 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:05.800 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:29.690 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:05.817 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:29.706 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:05.845 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:29.735 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:05.847 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:29.737 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:05.847 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:29.718 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:05.851 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:29.721 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:05.866 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:29.756 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:05.868 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:29.738 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:06.265 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-03-13T08:15:06.481 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:30.371 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:06.482 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:30.372 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:06.494 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:30.384 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:06.498 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:30.388 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:06.499 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:30.389 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:06.502 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:30.372 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:06.503 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:30.374 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:06.510 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:30.400 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:06.511 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:30.382 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:25.848 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:49.737 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:25.851 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:49.741 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:25.869 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:49.759 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:25.899 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:49.769 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:25.900 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:49.790 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:25.903 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:49.774 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:25.908 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:49.798 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:25.920 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:49.790 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:25.932 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:49.815 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:26.575 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:50.445 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:26.576 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:50.446 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:26.584 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:41:50.454 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:26.606 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:50.495 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:26.606 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:50.496 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:26.607 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:50.496 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:26.607 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:50.497 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:26.615 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:41:50.505 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:26.618 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:41:50.507 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:45.876 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:09.765 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:45.880 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:09.770 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:45.906 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:09.795 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:45.936 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:09.826 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:45.938 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:09.808 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:45.945 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:09.815 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:45.961 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:09.851 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:45.969 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:09.839 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:45.979 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:09.869 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:46.618 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:10.505 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:46.619 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:10.506 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:46.620 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:10.510 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:46.621 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:10.511 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:46.630 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:10.499 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:46.630 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:10.519 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:46.630 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:10.519 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:46.631 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:10.501 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:15:46.641 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:10.511 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:05.950 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:29.840 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:05.954 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:29.844 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:05.972 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:29.862 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:06.004 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:29.873 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:06.007 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:29.897 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:06.008 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:29.898 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:06.018 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:29.888 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:06.024 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:29.914 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:06.043 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:29.913 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:06.750 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:30.640 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:06.752 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:30.641 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:06.765 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:30.655 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:06.769 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:30.659 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:06.770 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:30.660 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:06.779 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:30.649 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:06.780 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:30.650 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:06.783 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:30.672 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:06.792 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:30.662 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:25.983 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:49.872 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:25.990 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:49.879 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:26.015 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:49.905 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:26.034 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:49.904 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:26.037 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:49.907 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:26.046 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:49.936 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:26.048 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:49.938 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:26.054 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:49.924 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:26.066 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:49.956 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:26.804 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:50.694 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:26.805 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:50.695 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:26.808 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:50.698 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:26.810 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:50.680 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:26.810 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:50.699 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:26.810 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:50.681 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:26.814 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:42:50.704 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:26.818 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:42:50.708 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:26.819 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:42:50.689 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.039 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:09.928 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.041 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:09.930 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.057 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:09.946 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.088 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:09.958 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.090 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:09.960 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.093 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:09.982 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.094 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:09.984 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.106 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:09.976 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.110 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:09.999 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.858 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:10.728 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.859 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:10.729 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.867 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:10.756 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.867 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:10.737 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.868 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:10.758 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.880 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:10.770 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.881 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:10.771 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.882 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:10.772 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:16:46.891 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:10.781 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.085 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:29.975 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.090 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:29.980 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.109 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:29.999 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.138 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:30.008 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.141 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:30.012 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.142 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:30.032 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.144 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:30.034 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.158 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:30.028 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.161 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:30.051 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.895 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:30.785 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.896 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:30.786 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.902 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:30.772 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.903 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:30.773 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.906 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:30.795 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.906 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:30.796 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.907 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:30.797 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.912 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:30.782 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:06.916 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:30.805 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:26.275 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:50.164 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:26.278 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:50.168 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:26.297 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:50.187 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:26.328 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:50.217 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:26.329 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:50.198 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:26.330 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:50.201 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:26.331 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:50.220 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:26.347 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:50.217 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:26.348 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:50.238 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:27.005 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:50.895 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:27.006 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:50.896 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:27.022 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:43:50.908 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:27.025 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:50.914 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:27.026 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:50.916 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:27.034 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:43:50.923 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:27.043 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:50.913 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:27.044 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:50.914 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:27.052 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:43:50.922 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:46.283 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:10.173 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:46.288 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:10.177 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:46.304 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:10.194 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:46.334 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:10.205 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:46.336 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:10.206 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:46.336 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:10.226 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:46.339 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:10.229 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:46.352 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:10.222 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:46.356 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:10.246 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:49.405 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:13.295 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:49.407 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:13.297 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:49.421 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:13.311 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:49.456 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:13.326 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:49.459 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:13.329 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:49.461 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:13.351 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:49.462 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:13.352 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:49.472 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:13.342 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:49.477 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:13.367 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:51.417 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:15.307 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:51.421 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:15.311 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:51.433 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:15.323 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:51.465 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:15.335 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:51.468 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:15.338 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:51.472 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:15.361 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:51.473 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:15.363 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:51.478 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:15.348 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:17:51.482 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:15.372 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:06.375 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:30.264 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:06.379 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:30.269 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:06.401 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:30.291 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:06.425 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:30.295 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:06.426 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:30.297 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:06.440 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:30.330 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:06.442 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:30.311 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:06.442 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:30.331 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:06.457 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:30.347 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:11.391 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:35.280 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:11.394 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:35.283 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:11.410 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:35.300 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:11.442 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:35.331 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:11.442 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:35.312 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:11.445 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:35.315 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:11.446 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:35.336 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:11.456 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:35.326 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:11.458 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:35.348 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:26.422 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:50.312 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:26.426 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:50.316 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:26.440 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:50.330 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:26.473 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:50.343 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:26.475 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:50.345 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:26.477 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:50.366 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:26.478 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:50.368 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:26.491 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:50.361 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:26.493 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:50.383 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:31.443 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:55.332 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:31.445 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:55.335 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:31.455 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:44:55.345 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:31.490 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:55.360 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:31.492 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:55.362 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:31.496 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:55.385 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:31.497 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:55.387 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:31.502 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:44:55.372 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:31.508 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:44:55.397 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:46.479 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:10.368 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:46.483 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:10.373 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:46.499 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:10.389 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:46.532 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:10.402 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:46.533 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:10.403 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:46.550 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:10.419 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:46.550 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:10.440 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:46.551 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:10.441 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:46.568 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:10.458 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:51.493 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:15.383 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:51.502 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:15.389 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:51.511 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:15.401 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:51.544 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:15.414 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:51.548 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:15.418 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:51.549 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:15.439 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:51.551 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:15.441 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:51.559 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:15.429 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:18:51.562 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:15.452 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:06.532 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:30.421 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:06.534 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:30.423 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:06.550 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:30.439 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:06.581 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:30.450 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:06.582 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:30.452 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:06.584 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:30.474 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:06.585 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:30.475 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:06.598 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:30.467 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:06.601 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:30.491 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:11.542 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:35.432 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:11.547 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:35.437 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:11.560 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:35.450 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:11.591 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:35.461 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:11.594 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:35.464 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:11.604 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:35.474 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:11.618 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:35.508 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:11.621 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:35.511 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:11.632 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:35.522 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:26.642 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:50.531 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:26.650 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:50.540 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:26.695 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:50.565 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:26.704 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:50.593 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:26.712 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:50.602 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:26.722 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:50.592 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:26.738 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:50.608 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:26.738 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:50.628 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:26.755 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:50.644 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:27.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-03-13T08:19:30.612 INFO:tasks.ceph.osd.5:Stopped 2024-03-13T08:19:30.612 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-13T08:19:30.612 INFO:teuthology.orchestra.run:waiting for 300 2024-03-13T08:19:30.624 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-03-13T08:19:30.616+0000 ac79640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 35245) UID: 0 2024-03-13T08:19:30.628 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-03-13T08:19:30.617+0000 ac79640 -1 osd.6 1305 *** Got signal Terminated *** 2024-03-13T08:19:31.674 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:55.563 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:31.678 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:55.568 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:31.691 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:45:55.581 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:31.726 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:55.596 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:31.727 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:55.616 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:31.727 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:55.597 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:31.728 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:55.618 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:31.741 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:45:55.611 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:31.741 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:45:55.631 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:33.179 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-03-13T08:19:41.704 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:05.593 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:41.708 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:05.598 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:41.722 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:05.612 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:41.756 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:05.626 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:41.757 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:05.627 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:41.757 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:05.647 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:41.758 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:05.648 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:41.769 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:05.639 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:41.770 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:05.660 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:51.729 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:15.618 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:51.732 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:15.622 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:51.750 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:15.640 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:51.777 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:15.647 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:51.778 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:15.648 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:51.783 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:15.672 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:51.786 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:15.676 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:51.795 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:15.665 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:51.799 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:15.689 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:19:55.018 INFO:tasks.ceph.osd.6:Stopped 2024-03-13T08:19:55.018 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-13T08:19:55.019 INFO:teuthology.orchestra.run:waiting for 300 2024-03-13T08:19:55.031 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T08:19:55.022+0000 ac79640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 35247) UID: 0 2024-03-13T08:19:55.035 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-13T08:19:55.024+0000 ac79640 -1 osd.7 1305 *** Got signal Terminated *** 2024-03-13T08:20:01.756 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:25.645 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:01.758 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:25.648 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:01.776 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:25.665 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:01.804 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:25.674 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:01.805 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:25.675 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:01.811 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:25.701 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:01.812 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:25.702 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:01.817 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:25.687 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:01.824 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:25.714 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:11.780 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:35.670 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:11.785 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:35.675 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:11.797 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:35.687 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:11.828 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:35.698 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:11.829 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:35.699 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:11.834 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:35.724 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:11.836 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:35.726 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:11.840 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:35.710 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:11.847 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:35.737 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:21.763 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:45.653 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:21.811 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:45.701 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:21.824 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:45.714 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:21.858 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:45.727 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:21.860 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:45.730 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:21.860 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:45.750 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:21.862 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:45.752 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:21.871 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:45.741 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:21.875 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:45.764 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:31.833 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:55.723 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:31.837 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:55.727 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:31.851 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:46:55.741 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:31.879 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:55.749 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:31.882 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:55.752 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:31.888 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:55.778 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:31.889 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:55.779 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:31.893 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:46:55.763 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:31.900 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:46:55.790 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:41.863 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:05.753 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:41.865 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:05.755 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:41.878 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:05.768 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:41.910 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:05.780 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:41.911 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:05.781 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:41.915 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:05.805 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:41.916 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:05.806 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:41.928 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:05.798 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:41.928 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:05.818 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:51.887 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:15.776 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:51.891 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:15.781 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:51.903 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:15.793 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:51.938 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:15.808 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:51.938 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:15.828 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:51.939 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:15.809 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:51.941 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:15.831 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:51.953 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:15.843 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:20:51.956 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:15.826 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:01.874 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:25.764 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:01.921 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:25.791 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:01.923 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:25.793 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:01.926 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:25.815 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:01.941 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:25.831 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:01.941 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:25.811 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:01.975 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:25.865 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:01.976 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:25.866 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:01.987 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:25.877 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:11.916 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:35.806 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:11.962 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:35.832 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:11.963 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:35.834 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:11.968 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:35.858 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:11.982 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:35.852 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:12.011 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:35.901 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:12.025 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:35.914 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:12.026 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:35.916 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:12.038 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:35.928 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:21.932 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:45.822 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:21.984 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:45.874 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:21.997 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:45.886 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:22.034 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:45.904 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:22.035 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:45.924 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:22.035 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:45.905 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:22.036 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:45.926 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:22.047 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:45.917 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:22.048 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:45.938 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:31.975 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:55.865 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.046 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:55.936 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.061 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:55.951 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.087 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:55.957 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.087 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:55.977 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.092 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:55.962 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.096 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:55.986 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.103 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:55.973 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.108 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:55.998 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.770 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:56.640 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.771 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:56.641 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.782 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:47:56.652 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.829 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:56.719 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.830 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:56.720 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.834 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:56.724 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.835 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:56.725 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.843 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:47:56.733 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:32.847 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:47:56.737 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:42.207 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:06.097 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:42.213 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:06.103 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:42.226 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:06.116 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:42.257 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:06.127 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:42.258 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:06.128 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:42.270 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:06.140 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:42.277 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:06.167 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:42.283 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:06.173 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:42.301 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:06.191 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:52.196 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:16.086 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:52.199 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:16.069 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:52.200 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:16.090 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:52.201 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:16.072 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:52.215 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:16.085 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:52.217 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:16.107 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:52.248 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:16.138 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:52.257 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:16.147 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:52.271 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:16.161 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:57.210 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:21.099 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:57.214 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:21.104 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:57.225 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:21.115 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:57.254 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:21.124 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:57.258 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:21.128 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:57.259 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:21.149 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:57.262 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:21.152 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:57.271 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:21.141 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:21:57.275 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:21.165 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:12.326 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:36.216 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:12.328 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:36.218 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:12.344 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:36.233 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:12.377 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:36.247 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:12.379 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:36.249 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:12.380 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:36.270 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:12.381 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:36.271 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:12.395 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:36.265 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:12.398 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:36.288 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:17.199 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:41.089 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:17.202 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:41.092 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:17.219 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:41.109 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:17.250 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:41.120 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:17.253 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:41.123 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:17.254 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:41.144 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:17.258 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:41.148 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:17.263 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:41.133 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:17.271 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:41.161 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:32.229 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:56.119 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:32.234 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:56.124 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:32.249 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:48:56.139 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:32.281 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:56.151 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:32.282 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:56.152 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:32.285 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:56.175 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:32.286 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:56.176 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:32.300 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:48:56.190 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:32.324 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:48:56.194 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:37.241 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:49:01.130 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:37.245 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:49:01.135 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:37.255 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:49:01.145 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:37.293 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:49:01.183 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:37.293 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:49:01.163 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:37.294 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:49:01.184 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:37.295 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:49:01.165 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:37.305 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:49:01.175 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:37.305 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:49:01.195 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:52.284 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:49:16.174 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:52.287 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:49:16.177 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:52.303 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:49:16.193 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:52.335 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:49:16.205 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:52.336 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:49:16.206 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:52.339 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:49:16.229 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:52.340 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:49:16.230 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:52.351 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:49:16.221 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:22:52.355 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:49:16.245 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:02.262 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:49:26.151 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:02.311 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:49:26.200 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:02.322 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:49:26.212 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:02.357 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:49:26.228 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:02.359 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:49:26.249 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:02.359 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:49:26.229 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:02.360 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:49:26.250 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:02.370 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:49:26.240 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:02.372 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:49:26.262 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:12.326 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:49:36.216 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:12.330 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:49:36.220 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:12.343 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:49:36.232 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:12.377 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:49:36.247 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:12.377 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:49:36.267 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:12.378 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:49:36.248 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:12.378 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:49:36.269 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:12.390 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:49:36.260 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:12.391 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:49:36.281 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:22.357 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:49:46.247 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:22.359 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:49:46.249 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:22.369 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:49:46.259 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:22.403 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:49:46.273 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:22.405 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:49:46.275 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:22.407 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:49:46.297 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:22.409 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:49:46.299 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:22.415 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:49:46.286 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:22.424 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:49:46.314 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:37.388 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:01.277 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:37.391 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:01.281 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:37.407 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:01.297 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:37.437 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:01.307 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:37.438 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:01.308 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:37.443 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:01.333 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:37.444 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:01.334 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:37.454 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:01.324 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:37.460 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:01.350 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:42.358 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:06.248 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:42.405 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:06.295 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:42.419 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:06.309 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:42.453 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:06.323 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:42.454 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:06.344 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:42.454 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:06.324 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:42.456 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:06.346 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:42.464 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:06.334 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:42.466 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:06.356 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:57.439 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:21.328 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:57.440 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:21.330 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:57.455 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:21.345 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:57.490 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:21.360 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:57.491 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:21.381 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:57.492 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:21.362 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:57.492 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:21.382 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:57.506 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:21.376 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:23:57.506 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:21.396 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:02.452 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:26.342 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:02.456 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:26.346 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:02.466 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:26.356 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:02.501 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:26.371 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:02.502 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:26.372 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:02.507 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:26.397 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:02.508 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:26.398 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:02.511 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:26.381 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:02.518 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:26.408 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:17.442 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:41.332 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:17.489 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:41.359 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:17.491 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:41.381 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:17.492 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:41.362 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:17.508 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:41.378 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:17.508 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:41.398 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:17.543 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:41.433 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:17.545 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:41.435 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:17.559 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:41.449 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:22.510 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:46.399 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:22.511 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:46.401 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:22.526 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:50:46.416 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:22.553 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:46.423 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:22.557 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:46.427 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:22.563 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:46.453 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:22.566 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:46.456 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:22.567 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:50:46.437 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:22.578 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:50:46.468 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:37.545 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:51:01.434 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:37.546 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:51:01.436 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:37.564 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:51:01.454 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:37.594 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:51:01.464 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:37.597 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:51:01.467 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:37.598 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:51:01.488 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:37.600 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:51:01.490 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:37.616 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:51:01.486 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:37.630 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:51:01.520 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:38.105 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:51:01.995 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:38.106 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:51:01.996 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:38.115 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:51:02.005 35240== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:38.154 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:51:02.044 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:38.155 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:51:02.045 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:38.165 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:51:02.055 35243== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:38.168 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:51:02.038 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:38.169 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:51:02.039 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:38.178 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:51:02.049 34705== Warning: unimplemented fcntl command: 1036 2024-03-13T08:24:54.086 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-13T08:24:54.086 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-13T08:24:54.086 INFO:teuthology.orchestra.run:waiting for 300 2024-03-13T08:24:54.088 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T08:24:54.087+0000 7fb6b1090640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mgr -f --cluster ceph -i y (PID: 35249) UID: 0 2024-03-13T08:24:54.088 INFO:tasks.ceph.mgr.y.smithi132.stderr:2024-03-13T08:24:54.087+0000 7fb6b1090640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-13T08:24:54.117 INFO:tasks.ceph.mgr.y:Stopped 2024-03-13T08:24:54.117 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-13T08:24:54.117 INFO:teuthology.orchestra.run:waiting for 300 2024-03-13T08:24:54.141 INFO:tasks.ceph.mgr.x:Stopped 2024-03-13T08:24:54.141 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-13T08:24:54.141 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-13T08:24:54.141 INFO:teuthology.orchestra.run:waiting for 300 2024-03-13T08:24:54.174 INFO:tasks.ceph.mon.a.smithi132.stderr:2024-03-13T08:24:54.161+0000 9f8e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 35209) UID: 0 2024-03-13T08:24:54.174 INFO:tasks.ceph.mon.a.smithi132.stderr:2024-03-13T08:24:54.163+0000 9f8e640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-13T08:24:55.041 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:51:18.930 35240== 2024-03-13T08:24:55.041 INFO:tasks.ceph.mon.a.smithi132.stderr:==00:00:51:18.930 35240== Exit program on first error (--exit-on-first-error=yes) 2024-03-13T08:24:55.070 INFO:tasks.ceph.mon.a.smithi132.stderr:daemon-helper: command failed with exit status 42 2024-03-13T08:24:58.258 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-03-13T08:25:00.243 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-03-13T08:25:00.243 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks/ceph.py", line 1449, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi132 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a" 2024-03-13T08:25:00.280 INFO:tasks.ceph.mon.a:Stopped 2024-03-13T08:25:00.280 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-13T08:25:00.280 INFO:teuthology.orchestra.run:waiting for 300 2024-03-13T08:25:00.296 INFO:tasks.ceph.mon.c.smithi132.stderr:2024-03-13T08:25:00.285+0000 9f8e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 35211) UID: 0 2024-03-13T08:25:00.297 INFO:tasks.ceph.mon.c.smithi132.stderr:2024-03-13T08:25:00.286+0000 9f8e640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2024-03-13T08:25:01.106 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:51:24.995 35243== 2024-03-13T08:25:01.106 INFO:tasks.ceph.mon.c.smithi132.stderr:==00:00:51:24.995 35243== Exit program on first error (--exit-on-first-error=yes) 2024-03-13T08:25:01.130 INFO:tasks.ceph.mon.c.smithi132.stderr:daemon-helper: command failed with exit status 42 2024-03-13T08:25:03.460 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-03-13T08:25:03.460 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-03-13T08:25:06.382 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-03-13T08:25:06.382 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks/ceph.py", line 1449, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi132 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c" 2024-03-13T08:25:06.382 INFO:tasks.ceph.mon.c:Stopped 2024-03-13T08:25:06.383 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-13T08:25:06.383 INFO:teuthology.orchestra.run:waiting for 300 2024-03-13T08:25:06.397 INFO:tasks.ceph.mon.b.smithi195.stderr:2024-03-13T08:25:06.386+0000 9f8e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 34701) UID: 0 2024-03-13T08:25:06.398 INFO:tasks.ceph.mon.b.smithi195.stderr:2024-03-13T08:25:06.388+0000 9f8e640 -1 mon.b@1(probing) e1 *** Got Signal Terminated *** 2024-03-13T08:25:07.245 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:51:31.115 34705== 2024-03-13T08:25:07.245 INFO:tasks.ceph.mon.b.smithi195.stderr:==00:00:51:31.115 34705== Exit program on first error (--exit-on-first-error=yes) 2024-03-13T08:25:07.266 INFO:tasks.ceph.mon.b.smithi195.stderr:daemon-helper: command failed with exit status 42 2024-03-13T08:25:08.562 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-03-13T08:25:08.563 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-03-13T08:25:12.484 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-03-13T08:25:12.484 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks/ceph.py", line 1449, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi195 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b" 2024-03-13T08:25:12.485 INFO:tasks.ceph.mon.b:Stopped 2024-03-13T08:25:12.485 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-13T08:25:12.486 DEBUG:teuthology.orchestra.run.smithi132:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | head -n 1 2024-03-13T08:25:12.531 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi132.front.sepia.ceph.com 2024-03-13T08:25:12.531 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-13T08:25:12.653 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi132.front.sepia.ceph.com 2024-03-13T08:25:12.653 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-13T08:25:12.708 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi132.front.sepia.ceph.com 2024-03-13T08:25:12.709 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-13T08:25:12.768 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi132.front.sepia.ceph.com 2024-03-13T08:25:12.768 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-13T08:25:12.843 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi195.front.sepia.ceph.com 2024-03-13T08:25:12.843 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-13T08:25:12.972 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi195.front.sepia.ceph.com 2024-03-13T08:25:12.973 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-13T08:25:13.038 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi195.front.sepia.ceph.com 2024-03-13T08:25:13.038 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-13T08:25:13.107 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi195.front.sepia.ceph.com 2024-03-13T08:25:13.107 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-13T08:25:13.189 INFO:tasks.ceph:Archiving mon data... 2024-03-13T08:25:13.191 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993/data/mon.a.tgz 2024-03-13T08:25:13.191 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-03-13T08:25:13.207 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.4n8jmDS2XV 2024-03-13T08:25:13.207 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.4n8jmDS2XV 2024-03-13T08:25:13.374 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.4n8jmDS2XV 2024-03-13T08:25:13.459 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.4n8jmDS2XV is 505KB 2024-03-13T08:25:13.529 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.4n8jmDS2XV 2024-03-13T08:25:13.543 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993/data/mon.c.tgz 2024-03-13T08:25:13.543 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-03-13T08:25:13.597 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.hCQ6TStIp4 2024-03-13T08:25:13.597 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.hCQ6TStIp4 2024-03-13T08:25:13.665 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-03-13T08:25:13.753 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.hCQ6TStIp4 2024-03-13T08:25:13.838 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.hCQ6TStIp4 is 507KB 2024-03-13T08:25:13.934 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.hCQ6TStIp4 2024-03-13T08:25:13.948 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993/data/mon.b.tgz 2024-03-13T08:25:13.949 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-03-13T08:25:13.963 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.FFNEw4fFBP 2024-03-13T08:25:13.963 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.FFNEw4fFBP 2024-03-13T08:25:14.138 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.FFNEw4fFBP 2024-03-13T08:25:14.222 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.FFNEw4fFBP is 509KB 2024-03-13T08:25:14.320 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.FFNEw4fFBP 2024-03-13T08:25:14.334 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-13T08:25:14.334 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-13T08:25:14.336 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-13T08:25:14.658 DEBUG:tasks.ceph:valgrind exception message: valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun 2024-03-13T08:25:14.913 ERROR:teuthology.util.scanner:Premature end of data in tag valgrindoutput line 3, line 50, column 1 (, line 50) 2024-03-13T08:25:14.915 INFO:tasks.ceph:Archiving crash dumps... 2024-03-13T08:25:14.917 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993/remote/smithi132/crash 2024-03-13T08:25:14.918 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-13T08:25:14.945 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993/remote/smithi195/crash 2024-03-13T08:25:14.946 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-13T08:25:14.972 INFO:tasks.ceph:Compressing logs... 2024-03-13T08:25:14.972 DEBUG:teuthology.orchestra.run.smithi132:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-13T08:25:14.990 DEBUG:teuthology.orchestra.run.smithi195:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-13T08:25:15.014 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40956.log 2024-03-13T08:25:15.014 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40230.log 2024-03-13T08:25:15.060 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-13T08:25:15.060 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34858.log 2024-03-13T08:25:15.060 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.7.log 2024-03-13T08:25:15.061 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.34858.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.5.log 2024-03-13T08:25:15.061 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/valgrind/osd.7.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.6.log 2024-03-13T08:25:15.061 INFO:teuthology.orchestra.run.smithi195.stderr: 55.1% -- replaced with /var/log/ceph/valgrind/osd.7.log.gz 2024-03-13T08:25:15.061 INFO:teuthology.orchestra.run.smithi195.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34858.log.gz 2024-03-13T08:25:15.061 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/valgrind/osd.5.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.b.log 2024-03-13T08:25:15.061 INFO:teuthology.orchestra.run.smithi195.stderr: 63.9% -- replaced with /var/log/ceph/valgrind/osd.5.log.gz 2024-03-13T08:25:15.061 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/valgrind/osd.6.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.4.log 2024-03-13T08:25:15.061 INFO:teuthology.orchestra.run.smithi195.stderr: 64.0% -- replaced with /var/log/ceph/valgrind/osd.6.log.gz 2024-03-13T08:25:15.061 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/valgrind/mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35059.log 2024-03-13T08:25:15.061 INFO:teuthology.orchestra.run.smithi195.stderr: 69.6% -- replaced with /var/log/ceph/valgrind/mon.b.log.gz 2024-03-13T08:25:15.061 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/valgrind/osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-13T08:25:15.061 INFO:teuthology.orchestra.run.smithi195.stderr: 64.0% -- replaced with /var/log/ceph/valgrind/osd.4.log.gz 2024-03-13T08:25:15.061 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.35059.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-13T08:25:15.061 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34978.log 2024-03-13T08:25:15.062 INFO:teuthology.orchestra.run.smithi195.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35059.log.gz 2024-03-13T08:25:15.062 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-13T08:25:15.062 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.34978.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-13T08:25:15.062 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35150.log 2024-03-13T08:25:15.062 INFO:teuthology.orchestra.run.smithi195.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34978.log.gz/var/log/ceph/ceph-osd.5.log: 2024-03-13T08:25:15.062 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34898.log 2024-03-13T08:25:15.062 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.35150.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-13T08:25:15.062 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.34898.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35150.log.gz 2024-03-13T08:25:15.062 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34898.log.gz 2024-03-13T08:25:15.062 INFO:teuthology.orchestra.run.smithi195.stderr: -- /var/log/ceph/ceph.log 2024-03-13T08:25:15.062 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35110.log 2024-03-13T08:25:15.062 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-mgr.x.log: 92.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-13T08:25:15.062 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-13T08:25:15.062 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.35110.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35110.log.gz 2024-03-13T08:25:15.063 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34938.log 2024-03-13T08:25:15.063 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mon.b.log: 87.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-13T08:25:15.063 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35018.log 2024-03-13T08:25:15.063 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.34938.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34938.log.gz 2024-03-13T08:25:15.063 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.35018.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35018.log.gz 2024-03-13T08:25:15.077 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.40956.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-13T08:25:15.077 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.40230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82043.log 2024-03-13T08:25:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41720.log 2024-03-13T08:25:15.078 INFO:teuthology.orchestra.run.smithi132.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40230.log.gz 2024-03-13T08:25:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109128.log 2024-03-13T08:25:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.82043.log: /var/log/ceph/ceph-client.admin.41720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39925.log 2024-03-13T08:25:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.109128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39838.log 2024-03-13T08:25:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39925.log: gzip -5 --verbose -- 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109128.log.gz 2024-03-13T08:25:15.078 INFO:teuthology.orchestra.run.smithi132.stderr: /var/log/ceph/ceph-client.admin.75190.log 2024-03-13T08:25:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82044.log 2024-03-13T08:25:15.078 INFO:teuthology.orchestra.run.smithi132.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39925.log.gz 2024-03-13T08:25:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.75190.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.39838.log.gz 2024-03-13T08:25:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41710.log 2024-03-13T08:25:15.078 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.82044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108623.log 2024-03-13T08:25:15.079 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82045.log 2024-03-13T08:25:15.079 INFO:teuthology.orchestra.run.smithi132.stderr: 90.5%/var/log/ceph/ceph-client.admin.108623.log: -- replaced with /var/log/ceph/ceph-client.admin.82043.log.gz 2024-03-13T08:25:15.079 INFO:teuthology.orchestra.run.smithi132.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.41720.log.gz 2024-03-13T08:25:15.079 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40198.log 2024-03-13T08:25:15.079 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.82045.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.108623.log.gz 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41710.log.gz 2024-03-13T08:25:15.079 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:25:15.079 INFO:teuthology.orchestra.run.smithi132.stderr:gzip 91.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.75190.log.gz -- 2024-03-13T08:25:15.079 INFO:teuthology.orchestra.run.smithi132.stderr: /var/log/ceph/ceph-client.admin.61064.log 2024-03-13T08:25:15.079 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40362.log 2024-03-13T08:25:15.079 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.40198.log: /var/log/ceph/ceph-client.admin.61064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39627.log 2024-03-13T08:25:15.079 INFO:teuthology.orchestra.run.smithi132.stderr: 91.8%/var/log/ceph/ceph-client.admin.40362.log: -- replaced with /var/log/ceph/ceph-client.admin.40956.log.gz 2024-03-13T08:25:15.079 INFO:teuthology.orchestra.run.smithi132.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40198.log.gz 90.5% -- replaced with /var/log/ceph/ceph-client.admin.82044.log.gz 2024-03-13T08:25:15.079 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:25:15.079 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mon.c.log 2024-03-13T08:25:15.080 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.2.log 2024-03-13T08:25:15.080 INFO:teuthology.orchestra.run.smithi132.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40362.log.gz 2024-03-13T08:25:15.080 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39627.log: /var/log/ceph/valgrind/mon.c.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.0.log 2024-03-13T08:25:15.080 INFO:teuthology.orchestra.run.smithi132.stderr: 69.6% -- replaced with /var/log/ceph/valgrind/mon.c.log.gz 2024-03-13T08:25:15.080 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/valgrind/osd.2.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.1.log 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39627.log.gz 2024-03-13T08:25:15.080 INFO:teuthology.orchestra.run.smithi132.stderr: 63.9% -- replaced with /var/log/ceph/valgrind/osd.2.log.gz 2024-03-13T08:25:15.080 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:25:15.080 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/valgrind/osd.0.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.82045.log.gz 2024-03-13T08:25:15.080 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mon.a.log 2024-03-13T08:25:15.080 INFO:teuthology.orchestra.run.smithi132.stderr: 63.9% -- replaced with /var/log/ceph/valgrind/osd.0.log.gz 2024-03-13T08:25:15.080 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/valgrind/osd.1.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61064.log.gz 2024-03-13T08:25:15.080 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.3.log 2024-03-13T08:25:15.080 INFO:teuthology.orchestra.run.smithi132.stderr: 63.9% -- replaced with /var/log/ceph/valgrind/osd.1.log.gz 2024-03-13T08:25:15.080 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/valgrind/mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41222.log 2024-03-13T08:25:15.080 INFO:teuthology.orchestra.run.smithi132.stderr: 69.5% -- replaced with /var/log/ceph/valgrind/mon.a.log.gz 2024-03-13T08:25:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/valgrind/osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108917.log 2024-03-13T08:25:15.081 INFO:teuthology.orchestra.run.smithi132.stderr: 64.0% -- replaced with /var/log/ceph/valgrind/osd.3.log.gz 2024-03-13T08:25:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:gzip/var/log/ceph/ceph-client.admin.41222.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41921.log 2024-03-13T08:25:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.108917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40496.log 2024-03-13T08:25:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39628.log 2024-03-13T08:25:15.081 INFO:teuthology.orchestra.run.smithi132.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108917.log.gz 2024-03-13T08:25:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.40496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40538.log 2024-03-13T08:25:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39876.log 2024-03-13T08:25:15.081 INFO:teuthology.orchestra.run.smithi132.stderr: 88.1%/var/log/ceph/ceph-client.admin.40538.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.40496.log.gz 2024-03-13T08:25:15.081 INFO:teuthology.orchestra.run.smithi132.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.108539.log 2024-03-13T08:25:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39876.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39628.log.gz 2024-03-13T08:25:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108497.log 2024-03-13T08:25:15.081 INFO:teuthology.orchestra.run.smithi132.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40538.log.gz 2024-03-13T08:25:15.081 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.108539.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39876.log.gz 2024-03-13T08:25:15.082 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41360.log 2024-03-13T08:25:15.082 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41238.log 2024-03-13T08:25:15.082 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.108497.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108539.log.gz 2024-03-13T08:25:15.082 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39108.log 2024-03-13T08:25:15.082 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41238.log: gzip -5 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108497.log.gz 2024-03-13T08:25:15.082 INFO:teuthology.orchestra.run.smithi132.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41977.log 2024-03-13T08:25:15.082 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39626.log 2024-03-13T08:25:15.082 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41977.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65333.log 2024-03-13T08:25:15.082 INFO:teuthology.orchestra.run.smithi132.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39108.log.gz 2024-03-13T08:25:15.082 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67762.log 2024-03-13T08:25:15.082 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.65333.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39626.log.gz 2024-03-13T08:25:15.082 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75192.log 2024-03-13T08:25:15.082 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.67762.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.41238.log.gz 2024-03-13T08:25:15.082 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose 90.8% -- replaced with /var/log/ceph/ceph-client.admin.65333.log.gz 2024-03-13T08:25:15.083 INFO:teuthology.orchestra.run.smithi132.stderr: -- /var/log/ceph/ceph-client.admin.40860.log 2024-03-13T08:25:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.75192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67763.log 2024-03-13T08:25:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.40860.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.75192.log.gz 2024-03-13T08:25:15.083 INFO:teuthology.orchestra.run.smithi132.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.67762.log.gz 2024-03-13T08:25:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41764.log 2024-03-13T08:25:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.67763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39404.log 2024-03-13T08:25:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41764.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.41222.log.gz 2024-03-13T08:25:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- 91.4% /var/log/ceph/ceph-client.admin.108875.log 2024-03-13T08:25:15.083 INFO:teuthology.orchestra.run.smithi132.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67763.log.gz 2024-03-13T08:25:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40454.log 2024-03-13T08:25:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.108875.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39404.log.gz 2024-03-13T08:25:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose 88.0% -- /var/log/ceph/ceph-client.admin.39446.log 2024-03-13T08:25:15.083 INFO:teuthology.orchestra.run.smithi132.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108875.log.gz 2024-03-13T08:25:15.083 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.40454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41092.log 2024-03-13T08:25:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39446.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.40454.log.gz 2024-03-13T08:25:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39622.log 2024-03-13T08:25:15.084 INFO:teuthology.orchestra.run.smithi132.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39446.log.gz 2024-03-13T08:25:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41092.log: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.40860.log.gz 2024-03-13T08:25:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61065.log 2024-03-13T08:25:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108959.log 2024-03-13T08:25:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39622.log: /var/log/ceph/ceph-client.admin.61065.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.41764.log.gz 2024-03-13T08:25:15.084 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39622.log.gz 2024-03-13T08:25:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108581.log 2024-03-13T08:25:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.108959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65334.log 2024-03-13T08:25:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.108581.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.108959.log.gz 2024-03-13T08:25:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108234.log 2024-03-13T08:25:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.65334.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108581.log.gz 2024-03-13T08:25:15.084 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-13T08:25:15.085 INFO:teuthology.orchestra.run.smithi132.stderr: 90.8% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65334.log.gz 2024-03-13T08:25:15.085 INFO:teuthology.orchestra.run.smithi132.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61065.log.gz 2024-03-13T08:25:15.085 INFO:teuthology.orchestra.run.smithi132.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41092.log.gz 2024-03-13T08:25:15.085 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.108234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35587.log 2024-03-13T08:25:15.085 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89453.log 2024-03-13T08:25:15.085 INFO:teuthology.orchestra.run.smithi132.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108234.log.gz/var/log/ceph/ceph-client.admin.35587.log: 2024-03-13T08:25:15.085 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109085.log 2024-03-13T08:25:15.085 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.89453.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35587.log.gz 2024-03-13T08:25:15.085 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41663.log 2024-03-13T08:25:15.085 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.109085.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109085.log.gz 2024-03-13T08:25:15.085 INFO:teuthology.orchestra.run.smithi132.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.41360.log.gz 2024-03-13T08:25:15.085 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42029.log 2024-03-13T08:25:15.085 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5/var/log/ceph/ceph-client.admin.41663.log: --verbose -- /var/log/ceph/ceph-client.admin.108329.log 2024-03-13T08:25:15.085 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.42029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41441.log 2024-03-13T08:25:15.086 INFO:teuthology.orchestra.run.smithi132.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41663.log.gz 90.8% -- replaced with /var/log/ceph/ceph-client.admin.89453.log.gz 2024-03-13T08:25:15.086 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:25:15.086 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.108329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41476.log 2024-03-13T08:25:15.086 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41754.log 2024-03-13T08:25:15.086 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41476.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108329.log.gz 2024-03-13T08:25:15.086 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41743.log 2024-03-13T08:25:15.086 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41754.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41754.log.gz 2024-03-13T08:25:15.086 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41306.log 2024-03-13T08:25:15.086 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41743.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.41921.log.gz 2024-03-13T08:25:15.086 INFO:teuthology.orchestra.run.smithi132.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.41476.log.gz 2024-03-13T08:25:15.086 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42063.log 2024-03-13T08:25:15.086 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82042.log 2024-03-13T08:25:15.086 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.42063.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.41743.log.gz 2024-03-13T08:25:15.086 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41908.log 2024-03-13T08:25:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.82042.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.41441.log.gz 2024-03-13T08:25:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109043.log 2024-03-13T08:25:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41908.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.82042.log.gz 2024-03-13T08:25:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41267.log 2024-03-13T08:25:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.109043.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.42029.log.gz 2024-03-13T08:25:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:gzip 95.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-13T08:25:15.087 INFO:teuthology.orchestra.run.smithi132.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39972.log 2024-03-13T08:25:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41267.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39881.log 2024-03-13T08:25:15.087 INFO:teuthology.orchestra.run.smithi132.stderr: -- replaced with /var/log/ceph/ceph-client.admin.109043.log.gz 2024-03-13T08:25:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108455.log 2024-03-13T08:25:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39881.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39972.log.gz 2024-03-13T08:25:15.087 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39881.log.gz 2024-03-13T08:25:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40034.log 2024-03-13T08:25:15.087 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.108455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35251.log 2024-03-13T08:25:15.088 INFO:teuthology.orchestra.run.smithi132.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108455.log.gz/var/log/ceph/ceph-client.admin.40034.log: 2024-03-13T08:25:15.088 INFO:teuthology.orchestra.run.smithi132.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40034.log.gz 2024-03-13T08:25:15.088 INFO:teuthology.orchestra.run.smithi132.stderr:gzip 90.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.41267.log.gz 2024-03-13T08:25:15.088 INFO:teuthology.orchestra.run.smithi132.stderr: --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-13T08:25:15.088 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40359.log/var/log/ceph/ceph-client.admin.35251.log: 2024-03-13T08:25:15.088 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39280.log 2024-03-13T08:25:15.088 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.40359.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.35251.log.gz 2024-03-13T08:25:15.088 INFO:teuthology.orchestra.run.smithi132.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40359.log.gz 2024-03-13T08:25:15.088 INFO:teuthology.orchestra.run.smithi132.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.41306.log.gz 2024-03-13T08:25:15.088 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89452.log 2024-03-13T08:25:15.088 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40160.log 2024-03-13T08:25:15.088 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.89452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41632.log 2024-03-13T08:25:15.088 INFO:teuthology.orchestra.run.smithi132.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39280.log.gz 2024-03-13T08:25:15.088 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.40160.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40160.log.gz 2024-03-13T08:25:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108286.log 2024-03-13T08:25:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41719.log 2024-03-13T08:25:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41632.log: /var/log/ceph/ceph-client.admin.108286.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108286.log.gz 2024-03-13T08:25:15.089 INFO:teuthology.orchestra.run.smithi132.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.89452.log.gz 2024-03-13T08:25:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40280.log 2024-03-13T08:25:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41796.log 2024-03-13T08:25:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.40280.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40280.log.gz 2024-03-13T08:25:15.089 INFO:teuthology.orchestra.run.smithi132.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41632.log.gz 2024-03-13T08:25:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37635.log 2024-03-13T08:25:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41709.log 2024-03-13T08:25:15.089 INFO:teuthology.orchestra.run.smithi132.stderr: 96.6% -- replaced with /var/log/ceph/ceph-client.admin.41977.log.gz 2024-03-13T08:25:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41796.log: /var/log/ceph/ceph-client.admin.37635.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37635.log.gz 2024-03-13T08:25:15.089 INFO:teuthology.orchestra.run.smithi132.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41719.log.gz 2024-03-13T08:25:15.089 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41532.log 2024-03-13T08:25:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42007.log 2024-03-13T08:25:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39617.log 2024-03-13T08:25:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.42007.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41532.log.gz 2024-03-13T08:25:15.090 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41485.log 2024-03-13T08:25:15.090 INFO:teuthology.orchestra.run.smithi195.stderr: 94.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-13T08:25:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39617.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39617.log.gz 2024-03-13T08:25:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39973.log 2024-03-13T08:25:15.104 INFO:teuthology.orchestra.run.smithi132.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.41709.log.gz 2024-03-13T08:25:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61063.log 2024-03-13T08:25:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39973.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39973.log.gz 2024-03-13T08:25:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39833.log 2024-03-13T08:25:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.61063.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.61063.log.gz 2024-03-13T08:25:15.104 INFO:teuthology.orchestra.run.smithi132.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.41796.log.gz 2024-03-13T08:25:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-13T08:25:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38891.log 2024-03-13T08:25:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-osd.3.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39833.log.gz 2024-03-13T08:25:15.104 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39362.log 2024-03-13T08:25:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.38891.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38891.log.gz 2024-03-13T08:25:15.105 INFO:teuthology.orchestra.run.smithi132.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.41485.log.gz 2024-03-13T08:25:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41151.log 2024-03-13T08:25:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108791.log 2024-03-13T08:25:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41151.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39362.log.gz 2024-03-13T08:25:15.105 INFO:teuthology.orchestra.run.smithi132.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.42007.log.gz 2024-03-13T08:25:15.105 INFO:teuthology.orchestra.run.smithi132.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.41908.log.gz 2024-03-13T08:25:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36592.log 2024-03-13T08:25:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.108791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109168.log 2024-03-13T08:25:15.105 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.36592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40911.log 2024-03-13T08:25:15.105 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108791.log.gz/var/log/ceph/ceph-client.admin.109168.log: 2024-03-13T08:25:15.105 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36592.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89454.log 2024-03-13T08:25:15.105 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:25:15.105 INFO:teuthology.orchestra.run.smithi132.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109168.log.gz 2024-03-13T08:25:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.40911.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.41151.log.gz 2024-03-13T08:25:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39625.log 2024-03-13T08:25:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67764.log 2024-03-13T08:25:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.89454.log: /var/log/ceph/ceph-client.admin.39625.log: gzip -5 --verbose 91.4% -- replaced with /var/log/ceph/ceph-client.admin.40911.log.gz 2024-03-13T08:25:15.106 INFO:teuthology.orchestra.run.smithi132.stderr: -- /var/log/ceph/ceph-client.admin.41536.log 2024-03-13T08:25:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.67764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40865.log 2024-03-13T08:25:15.106 INFO:teuthology.orchestra.run.smithi132.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39625.log.gz 2024-03-13T08:25:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41536.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.42063.log.gz 2024-03-13T08:25:15.106 INFO:teuthology.orchestra.run.smithi132.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.89454.log.gz 2024-03-13T08:25:15.106 INFO:teuthology.orchestra.run.smithi132.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.67764.log.gz 2024-03-13T08:25:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-13T08:25:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108665.log/var/log/ceph/ceph-client.admin.40865.log: 2024-03-13T08:25:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82041.log 2024-03-13T08:25:15.106 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.108665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108707.log 2024-03-13T08:25:15.107 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.82041.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108665.log.gz 2024-03-13T08:25:15.107 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40822.log 2024-03-13T08:25:15.107 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.108707.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108707.log.gz 2024-03-13T08:25:15.107 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108749.log 2024-03-13T08:25:15.108 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.40822.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.82041.log.gz 2024-03-13T08:25:15.108 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35771.log 2024-03-13T08:25:15.108 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.108749.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108749.log.gz 2024-03-13T08:25:15.108 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75189.log 2024-03-13T08:25:15.108 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.35771.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35771.log.gz 2024-03-13T08:25:15.109 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41102.log 2024-03-13T08:25:15.111 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.75189.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.40865.log.gz 2024-03-13T08:25:15.112 INFO:teuthology.orchestra.run.smithi132.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.75189.log.gz 2024-03-13T08:25:15.112 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39320.log 2024-03-13T08:25:15.112 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41026.log 2024-03-13T08:25:15.112 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39320.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39320.log.gz 2024-03-13T08:25:15.112 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108371.log 2024-03-13T08:25:15.125 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41026.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-13T08:25:15.126 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.108371.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108371.log.gz 2024-03-13T08:25:15.132 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39624.log 2024-03-13T08:25:15.138 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-osd.0.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.40822.log.gz 2024-03-13T08:25:15.145 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39224.log 2024-03-13T08:25:15.146 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39624.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39624.log.gz 2024-03-13T08:25:15.152 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35659.log 2024-03-13T08:25:15.153 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39224.log: 92.0% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39224.log.gz 2024-03-13T08:25:15.153 INFO:teuthology.orchestra.run.smithi132.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41536.log.gz 2024-03-13T08:25:15.167 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-13T08:25:15.168 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.35659.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35659.log.gz 2024-03-13T08:25:15.169 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41744.log 2024-03-13T08:25:15.183 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-13T08:25:15.183 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41744.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41744.log.gz 2024-03-13T08:25:15.188 INFO:teuthology.orchestra.run.smithi132.stderr: 93.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-13T08:25:15.188 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109001.log 2024-03-13T08:25:15.201 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42026.log 2024-03-13T08:25:15.202 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.109001.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109001.log.gz 2024-03-13T08:25:15.208 INFO:teuthology.orchestra.run.smithi132.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.41026.log.gz 2024-03-13T08:25:15.211 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108413.log 2024-03-13T08:25:15.224 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.42026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39179.log 2024-03-13T08:25:15.225 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.108413.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108413.log.gz 2024-03-13T08:25:15.231 INFO:teuthology.orchestra.run.smithi132.stderr: 93.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40848.log 2024-03-13T08:25:15.233 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39179.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39179.log.gz 2024-03-13T08:25:15.233 INFO:teuthology.orchestra.run.smithi132.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42026.log.gz 2024-03-13T08:25:15.233 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40354.log 2024-03-13T08:25:15.247 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.40848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108833.log 2024-03-13T08:25:15.247 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.40354.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40354.log.gz 2024-03-13T08:25:15.262 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41278.log 2024-03-13T08:25:15.264 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.108833.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108833.log.gz 2024-03-13T08:25:15.264 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29430.log 2024-03-13T08:25:15.271 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39620.log 2024-03-13T08:25:15.271 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph.tmp-client.admin.29430.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29430.log.gz 2024-03-13T08:25:15.278 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41973.log 2024-03-13T08:25:15.279 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.39620.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39620.log.gz 2024-03-13T08:25:15.284 INFO:teuthology.orchestra.run.smithi132.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.41278.log.gz 2024-03-13T08:25:15.298 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40907.log 2024-03-13T08:25:15.302 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41973.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.41973.log.gz 2024-03-13T08:25:15.315 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40999.log 2024-03-13T08:25:15.327 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.40907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41388.log 2024-03-13T08:25:15.336 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.40999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41858.log 2024-03-13T08:25:15.338 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41388.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.40848.log.gz 2024-03-13T08:25:15.343 INFO:teuthology.orchestra.run.smithi132.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.41388.log.gz 2024-03-13T08:25:15.351 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41008.log 2024-03-13T08:25:15.368 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41858.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-13T08:25:15.374 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.41008.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.41008.log.gz 2024-03-13T08:25:15.376 INFO:teuthology.orchestra.run.smithi132.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.41858.log.gz 2024-03-13T08:25:15.401 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-mon.c.log: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.40907.log.gz 2024-03-13T08:25:15.675 INFO:teuthology.orchestra.run.smithi132.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-13T08:25:15.814 INFO:teuthology.orchestra.run.smithi132.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.41102.log.gz 2024-03-13T08:25:16.742 INFO:teuthology.orchestra.run.smithi132.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.40999.log.gz 2024-03-13T08:25:16.790 INFO:teuthology.orchestra.run.smithi195.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-13T08:25:17.575 INFO:teuthology.orchestra.run.smithi132.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-13T08:25:19.077 INFO:teuthology.orchestra.run.smithi132.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-13T08:25:22.015 INFO:teuthology.orchestra.run.smithi132.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-13T08:25:22.735 INFO:teuthology.orchestra.run.smithi132.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-13T08:25:23.017 INFO:teuthology.orchestra.run.smithi195.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-13T08:25:23.211 INFO:teuthology.orchestra.run.smithi195.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-13T08:25:23.489 INFO:teuthology.orchestra.run.smithi132.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-13T08:25:23.577 INFO:teuthology.orchestra.run.smithi132.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-13T08:25:23.579 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-13T08:25:23.579 INFO:teuthology.orchestra.run.smithi132.stderr:real 0m8.577s 2024-03-13T08:25:23.579 INFO:teuthology.orchestra.run.smithi132.stderr:user 0m39.385s 2024-03-13T08:25:23.579 INFO:teuthology.orchestra.run.smithi132.stderr:sys 0m1.371s 2024-03-13T08:25:24.473 INFO:teuthology.orchestra.run.smithi195.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-13T08:25:25.141 INFO:teuthology.orchestra.run.smithi195.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-13T08:25:25.144 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-13T08:25:25.144 INFO:teuthology.orchestra.run.smithi195.stderr:real 0m10.115s 2024-03-13T08:25:25.144 INFO:teuthology.orchestra.run.smithi195.stderr:user 0m36.399s 2024-03-13T08:25:25.144 INFO:teuthology.orchestra.run.smithi195.stderr:sys 0m1.034s 2024-03-13T08:25:25.144 INFO:tasks.ceph:Archiving logs... 2024-03-13T08:25:25.145 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993/remote/smithi132/log 2024-03-13T08:25:25.146 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-13T08:25:30.500 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993/remote/smithi195/log 2024-03-13T08:25:30.501 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-13T08:25:35.043 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks/ceph.py", line 329, in valgrind_post yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks/ceph.py", line 1449, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun 2024-03-13T08:25:35.044 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-13T08:25:35.053 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks/ceph.py", line 329, in valgrind_post yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks/ceph.py", line 1449, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_b7c579c9c3ed531070a6aa96f74f80b53f80de32/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun 2024-03-13T08:25:35.054 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-13T08:25:35.054 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-13T08:25:35.057 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-13T08:25:35.137 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-13T08:25:35.146 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-13T08:25:35.147 DEBUG:teuthology.orchestra.run.smithi132:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-13T08:25:35.149 DEBUG:teuthology.orchestra.run.smithi195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-13T08:25:35.162 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2024-03-13T08:25:35.163 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 1: ntpq: command not found 2024-03-13T08:25:35.167 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-13T08:25:35.167 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2024-03-13T08:25:35.167 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 +597us[ +597us] +/- 111ms 2024-03-13T08:25:35.167 INFO:teuthology.orchestra.run.smithi195.stdout:^* hv02.front.sepia.ceph.com 3 6 377 46 -461us[ -469us] +/- 62ms 2024-03-13T08:25:35.167 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 +127us[ +119us] +/- 72ms 2024-03-13T08:25:35.167 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T08:25:35.168 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-13T08:25:35.168 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-03-13T08:25:35.168 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 43 +609us[ +609us] +/- 111ms 2024-03-13T08:25:35.168 INFO:teuthology.orchestra.run.smithi132.stdout:^* hv02.front.sepia.ceph.com 3 6 377 43 -444us[ -418us] +/- 62ms 2024-03-13T08:25:35.168 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +143us[ +143us] +/- 72ms 2024-03-13T08:25:35.168 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T08:25:35.169 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-13T08:25:35.182 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-13T08:25:35.183 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-13T08:25:35.191 DEBUG:teuthology.orchestra.run.smithi132:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-13T08:25:35.283 DEBUG:teuthology.orchestra.run.smithi195:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-13T08:25:35.348 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-13T08:25:35.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T08:25:35.380 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-13T08:25:35.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T08:25:35.409 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-13T08:25:35.418 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-13T08:25:35.427 INFO:teuthology.task.internal:Duration was 3993.108498 seconds 2024-03-13T08:25:35.427 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-13T08:25:35.436 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-13T08:25:35.436 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-13T08:25:35.439 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-13T08:25:35.482 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-13T08:25:35.492 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-13T08:25:35.813 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-13T08:25:35.813 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi132.front.sepia.ceph.com 2024-03-13T08:25:35.813 DEBUG:teuthology.orchestra.run.smithi132:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-13T08:25:35.832 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi195.front.sepia.ceph.com 2024-03-13T08:25:35.833 DEBUG:teuthology.orchestra.run.smithi195:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-13T08:25:35.853 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-13T08:25:35.853 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-13T08:25:35.875 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-13T08:25:35.920 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-13T08:25:35.921 DEBUG:teuthology.orchestra.run.smithi132:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-13T08:25:35.945 DEBUG:teuthology.orchestra.run.smithi195:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-13T08:25:36.061 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-13T08:25:36.071 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-13T08:25:36.072 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-13T08:25:36.098 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-13T08:25:36.124 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-13T08:25:36.134 DEBUG:teuthology.orchestra.run.smithi132:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-13T08:25:36.141 DEBUG:teuthology.orchestra.run.smithi195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-13T08:25:36.164 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = core 2024-03-13T08:25:36.189 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = core 2024-03-13T08:25:36.208 DEBUG:teuthology.orchestra.run.smithi132:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-13T08:25:36.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T08:25:36.234 DEBUG:teuthology.orchestra.run.smithi195:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-13T08:25:36.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T08:25:36.262 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-13T08:25:36.272 INFO:teuthology.task.internal:Transferring archived files... 2024-03-13T08:25:36.273 DEBUG:teuthology.misc:Transferring archived files from smithi132:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993/remote/smithi132 2024-03-13T08:25:36.273 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-13T08:25:36.330 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993/remote/smithi195 2024-03-13T08:25:36.330 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-13T08:25:36.365 INFO:teuthology.task.internal:Removing archive directory... 2024-03-13T08:25:36.366 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-13T08:25:36.368 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-13T08:25:36.413 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-13T08:25:36.426 INFO:teuthology.task.internal:Not uploading archives. 2024-03-13T08:25:36.426 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-13T08:25:36.437 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-13T08:25:36.437 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-13T08:25:36.439 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-13T08:25:36.451 INFO:teuthology.orchestra.run.smithi132.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 13 08:25 /home/ubuntu/cephtest 2024-03-13T08:25:36.467 INFO:teuthology.orchestra.run.smithi195.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 13 08:25 /home/ubuntu/cephtest 2024-03-13T08:25:36.468 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-13T08:25:36.478 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-13T08:25:36.488 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: Exception: valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun 2024-03-13T08:25:36.489 INFO:teuthology.nuke:Checking targets against current locks 2024-03-13T08:25:36.515 DEBUG:teuthology.nuke:shortname: smithi132 2024-03-13T08:25:36.515 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-13T08:25:36.538 DEBUG:teuthology.nuke:shortname: smithi195 2024-03-13T08:25:36.538 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-13T08:25:36.541 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-13 07:10:22.440330', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEFZB1Zl15oaEhsQoQkm4aIPLf9pprFtx9T0xmH3GkM7BJHh9G1TANIlJ/xMCqXsZ7bEm0hx6mFdZpYK3/Lq3jE='} 2024-03-13T08:25:36.563 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-12_18:29:22-rados-wip-yuri8-testing-2024-03-11-1138-distro-default-smithi/7594993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-13 07:10:22.441691', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3eaO3daDzXfRTyTOLae0YzBR4WELsoKJuVa+GGGYjFqxjdnHE6e5I67oohVXUtMNTEGwJME1e7kHpeYpVSJXs='} 2024-03-13T08:25:36.594 INFO:teuthology.orchestra.console.smithi132:Power off 2024-03-13T08:25:36.594 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-13T08:25:36.618 DEBUG:teuthology.orchestra.console.smithi132:power off output: Chassis Power Control: Down/Off 2024-03-13T08:25:36.618 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T08:25:36.633 INFO:teuthology.orchestra.console.smithi195:Power off 2024-03-13T08:25:36.633 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-13T08:25:36.642 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-03-13T08:25:36.655 DEBUG:teuthology.orchestra.console.smithi195:power off output: Chassis Power Control: Down/Off 2024-03-13T08:25:36.656 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T08:25:36.678 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2024-03-13T08:25:40.644 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T08:25:40.679 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T08:25:40.754 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-03-13T08:25:40.788 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2024-03-13T08:25:44.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-03-13T08:25:44.756 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T08:25:44.790 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T08:25:44.866 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2024-03-13T08:25:44.899 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2024-03-13T08:25:44.968 INFO:teuthology.orchestra.console.smithi132:Power off completed 2024-03-13T08:25:48.902 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T08:25:49.014 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off 2024-03-13T08:25:49.114 INFO:teuthology.orchestra.console.smithi195:Power off completed 2024-03-13T08:25:49.199 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~5s 2024-03-13T08:25:49.219 INFO:teuthology.run:Summary data: description: rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} duration: 3993.1084983348846 failure_reason: 'valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun' flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2024-03-13T08:25:49.219 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T08:25:49.298 INFO:teuthology.run:FAIL